DOI: 10.5281/zenodo.15592132
title: Bibliographic records on Polish samizdat books

description: This dataset is an outcome of a retro-digitization and digital curation project, which builds a bibliography of unlicensed non-serial prints circulating in Poland in 1976-1989, created by the National Library of Poland (cf. the acknowledgements below).

 
description: The dataset consists of several CSV files exported from Nodegoat,  "a web-based research environment for the humanities" which combines a relational database with data modeling and spatial and temporal visualisation tools.

The CSV files correspond to main groups of objects of the same type ("object types") set up in Nodegoat. The PNG file "Nodegoat data model" represents the data structure of each of these types of object, as well as the interconnections between these structures. 

WORKS (4885 objects) - contains descriptions of works, their titles and creators. It connects with other objects through the unique identifier Work_ID.

PERSONS (2848 objects) - contains descriptions of persons: names and pseudonyms, sex labels, dates and places of birth and death,  as well as VIAF and Wikidata identifiers. It connects with other objects through the unique identifier Per_ID.

CORPORATE ENTITIES (1935 objects) - contains descriptions of corporate entities: name and acronym, location, and relationship to other corporate entities. It connects with other objects through the unique identifier Corp_ID.

GAZETTEER (1142 objects) - contains descriptions of locations: Polish and English names, Wikidata and Geonames IDs, country codes, and coordinates. It connects with other objects through the unique Wikidata identifier. 

SERIES (289 objects) - contains descriptions of book series, titles and creators. It connects with other objects through the unique identifier Series_ID.

RECORDS (6512 objects ): contains descriptions of bibliographic records represented in the dataset: title year of publication, places of publication, languages (of text, original work and source of translation). In addition it contains descriptions of following sub-objects:





Contained work: represented works and its authors, contributors, translators and corporate creators




Edition: publisher and printer




Secondary samizdat source (reprints, translations etc): represented works, source publishers, source journals and journal issues




Series: labels of book series, subseries and series numbers



Each record has the unique identifier Record_ID.

GENERAL NOTE: Nodegoat is a relational database. The CSV files represent a “flat” rendering of that database. In each CSV file there are more rows than object descriptions, because in case of multiple object/sub-object descriptions, each description is paired with the object in a separate row (e.g. if the one record represents two works, each work ID is paired with the related record ID in a separate row).
description: Data Curation Pipeline

Data scraping





We received the scraped data from the National Library (BN) database in the form of a TXT file which my colleague Cezary Rosiński then parsed and tabularized to enable further processing.



Data modeling





The BN records are structured according to the MARC BN format, in use 1986-1998, before BN switched to international standards of bibliographic description (MARC21, Dublin Core). Based on the available documentation, we analysed which field contains what kind of information.




Based on that analysis we designed our data environment in Nodegoat. We created object types, that is both data models of particular entities populating our dataset, which at the level of the data group objects of the same type. For instance, our data model for the object type “Persons” specifies a number of object descriptions (e.g. name, pseudonym, sex label) and sub-object descriptions (dates and places of birth and death). On the level of the data, the objects described according to the type “Persons” are grouped together, so you can also understand it as the authority file of persons.




The data types/authority files used in the project are described in the technical info section. These are: persons, corporate entities, places (the gazetter), works, series and records. We also created classifications (e.g. language, country codes, sex label, role). The “record” type of object (corresponding to the bibliographic records in the BN dataset) connects together the authorities and the classifications with stand-alone data.




Each type of object has been matched against the MARC BN format metadata fields. In particular, we had to select which kind of information we want to include, taking into account its prospective usefulness for data-driven research, and then decide where that information fits in our data model, taking into account various features of the Nodegoat research environment (e.g. data that we wanted to use for spatial and temporal visualisation, had to be mapped on the sub-object level).




This process, presented here in a linear way, has been in fact iterative -  as we processed the BN dataset, the original design has been modified, adding, subtracting or moving around different elements of the data model. That included adding the authority file of works as a separate type of object, and creating  the “secondary samizdat source” sub-object of the record object, which integrates data from various note fields that contain information about the sources of translations, reprints, etc., into structured data (more on that below).



Automated Named Entities Recognition (NER)





Working in Python, Cezary extracted the named entities (persons, places and corporate entities) from the dataset. 




Next, for the purpose of disambiguation, he has written the code using probability algorithms and machine learning to match named entities against the available Linked Open Data (LOD) resources (VIAF and Wikidata in case of persons, Wikidata and geonames in case of places). This was also an iterative process, in which each round consisted in assessing the accuracy of the matchings (me) and modifying the code (Cezary). It has to be stressed that this process took place before the LLM revolution!



Semi-manual data normalization and enrichment





Once it became clear that the automated NER iterations yield diminishing returns, I proceeded with further data reconciliation identifying both false negatives (different names of a single entity identified as different entities) and false positives (distinct entities identified as a single entity). This was both a technical task, and a research task.




The technical dimension consisted in identifying different name variants of an entity: full names, abbreviated names and acronyms, and deciding whether those different variants belong together. For instance, in case of corporate entities, SOWA (“owl” in English) can stand for the acronym of Studencka Agencja Wydawnicza (Szczecin) or Studencka Oficyna Wydawnicza (Warsaw), or for the proper name of the Sowa publishing house (Bydgoszcz). To sort that out I was using various features of MS Excel (data filtering and formulas) and Open Refine (Cluster and Edit function) to manipulate the data.




I sometimes made these decisions based on inspecting adjacent data alone (e.g. similar name variants relating to a single place of publishing suggest the same entity), but often I had to do additional fact-checking,  using different documentary resources on Polish dissent (encyclopedias, bibliographies, dictionaries of pseudonyms etc, but also Wikipedia). For example, I could consult the entries about samizdat publishers in the Encyclopedia of Solidarity to verify which works were published by which publisher. Or I could also consult the pseudonym dictionary “Kto był kim w drugim obiegu” [“The Samizdat Who Was Who”] (IBL PAN, 1995), to identify the unresolved pseudonyms.




For the authority files of persons and places, the next step was the verification of LOD references in order to increase the usability of the dataset, but also, in case of Wikidata, to be able to import the missing metadata (e.g. dates and places of birth and death) and to make further data enrichment possible in the future.




I introduced an object type “Works” that does not match the original data model, but which was key to make the dataset richer and more complex. This type of object connects different variants of the same cultural creation, for instance translations of the same work appearing under different titles, or reprint of a fragment (e.g. chapter) of a work. As above, here as well data reconciliation involved both data manipulation and research.




In the Record type of object, I introduced the sub-object “secondary samizdat source” which captures the provenance information about works that were not samizdat-born, e.g. reprints of other samizdat works, pre-1945 works published in Poland, translations etc. In the BN dataset, that provenance information was included in a couple of different unstructured note fields, which I had to parse. Also the information in these fields was often incomplete (e.g. missing the publisher or the year of publication ), and required a lot of additional fact checking. Some of “secondary samizdat” was not identified as such at all, but could be identified when filtering by language of original/translation source (not Polish) or extrapolated by examining records that share the same works with the records identified as secondary samizdat. In some cases the research yielded uncertain results (e.g. which of many the editions of Orwell’s 1984 was used for translation). This is why some of these sub-object descriptions are identified as “approximate”. 



Data import and revision





Once I was satisfied with the results of the data normalization, I proceeded with the first data import into Nodegoat.




Next, I revised the dataset for both individual errors of description, modeling mistakes and usability (whether the data work well in visualisations).



What to expect in the future?

This is the first, but certainly not the last release of this dataset. Here is a list of planned enhancements: 





Persons authority file - data enrichment using “occupation” element from Wikidata




Processing the BN subject headings file




Processing of the physical description field from the original dataset
description: Acknowledgements

This dataset builds on the  bibliography of samizdat non-serials circulating in Poland in 1976-1989, created by  Grażyna Fedorowicz, Krystyna Gromadzińska and Maria Kaczyńska for the National Library of Poland. The print version,  “Bibliografia podziemnych druków zwartych z lat 1976–1989” was published in 1995, followed by the bibliographic database released in 2002, which serves as the groundwork for the current project. Working in the Nodegoat research environment, I have extensively curated, updated and enriched the dataset to serve the purposes of digital humanities data-driven research.

My colleague Cezary Rosiński provided technical assistance in the early stages of the data curation process. The first data release has been generously supported by the Polish National Agency for Academic Exchange (NAWA, grant  BJP/PON/2024/1/00009).