Proceedings of the 10th Conference of theJapanese Association of Digital Humanities

JADH2020“A New Decade in Digital Scholarship:”

Microcosms and Hubs

November 20-22, 2020

JADH2020

Proceedings of JADH Conference, vol. 2020

“A New Decade in Digital Scholarship: Microcosms and Hubs”
https://jadh2020.lang.osaka-u.ac.jp
November 20-22, 2020

Edited by Bor Hodošček
Copyright © 2020 by the Japanese Association of Digital Humanities
Co-sponsors:
Japan Society for Digital Archives
Japan Association for English Corpus Studies
Japan Association for East Asian Text Processing
The Mathematical Linguistic Society
IPSJ SIG Computers and Humanities
Japan Society of Information and Knowledge
Japan Art Documentation Society
Published by:
Graduate School of Language and Culture, Osaka University
1-8 Machikaneyama-cho, Toyonaka, Osaka 560-0043 Japan
https://www.lang.osaka-u.ac.jp/lc/en/
Online edition:
Print edition:

ISSN 2432-3144
ISSN 2432-3187

Contents
PlenaryPanel

The Poetics of Scale in Microcosmic/Macrocosmic World Literature: Perspective, Method, andLogic in Zooming-in and Zooming out of World Literature
Youngmin Kim (Dongguk University, Hangzhou Normal Universy) . . . . . . . . . . . . .
The Regional Turn in Digital Humanities in India: Decolonizing Digital Knowledge Infrastructures
Nirmala Menon (Indian Institute of Technology) . . . . . . . . . . . . . . . . . . . . . . .

Tracing playful media practice: Challenges for Theory, Methodology, and Infrastructure
Martin Roth (Chair), Kazufumi Fukuda, Zoltan Kacsuk, Akito Inoue . . . . . . . . . . . . .
Long Papers

1
4

7

Reviewing Digital Literature through Digital Literature
David Thomas Henry Wright . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
“He triumphs; maybe, we shall stand alone”
Iku Fujita . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Linked-Potter: an example of ontology for the study of the evolution of literature and reading
communities
Federico Pianzola . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Multinomial Mixed-Effects Models and Linguistic Variation: Competitions among Japanese SubjectHonorific Constructions
Akitaka Yamada . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Digital Editing of Japanese Buddhist Texts from the Manuscript to TEI and Database Building
Gaétan Rappo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Between óneiros and pragma: the social context of video game humor
Artur Skweres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Big Data as Archive: Data Centers and Digital Traces as Future Historical Artifacts
Yennie Jun, Javier Cha . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Short Papers

Posters

Digital Public History during the COVID-19 Pandemic: Discussion on the Corona Archive @ Kansai University
Nobuhiko Kikuchi, Keiichi Uchida, Tadakatsu Okada, Takefumi Hayashi, Takao Fujita, Satoshi
Ninomiya, So Miyagawa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Re-interpreting the Paintings and Poems of the Lotus Sutra through a Co-occurrence Network
Aiko Aida . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Computer Vision-based Comparison of Woodblock-printed Books and its Application to Japanese
Pre-modern Text, Bukan
Thomas Leyh, Asanobu Kitamoto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Clarification of boundaries and criteria for periodisation in Beethoven’s career
Daisuke Miki, Akihiro Kawase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Automated Face Detection for Pre-modern Japanese Artworks using Deep Neural Networks
Alexis Mermet, Asanobu Kitamoto, Chikahiko Suzuki, Akira Takagishi . . . . . . . . . . . 65

ii

GIS Visualisation for the Administrative Document Catalogue of the Government-General of Taiwan
Hajime Murai, Toshio Kawashima . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Educational utilization of digital archive -Inquiry based curation learning in on-line educationMasao Oi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
“Noh as Intermedia”/Intermedia Analyses of Noh Theater Plays
Takanori Fujita, Jarosław Kapuściński, François Rose, Scott Bailey, Peter Broadwell, Simon
Wiles, Glen Worthey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A fundamental study of effective visualization and description of longitudinal changes of classical Japanese poetic vocabulary
Hilofumi Yamamoto, Bor Hodošček . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Analysis of difference between male and female facial expressions in Japanese picture scrolls
using GM Method with IIIF Curation Platform
Chikahiko Suzuki, Akira Takagishi, Alexis Mermet, Asanobu Kitamoto, Jun Homma . . . .

71
77
82
87
90

JADH2020

Plenary - (1)The Poetics of Scale in Microcosmic/Macrocosmic
World Literature
Perspective, Method, and Logic in Zooming-in and Zooming out of World
Literature
Youngmin Kim1
1

Distinguished Research Professor Emeritus, Dongguk University(youngm@dongguk.edu)
2Jack Ma Chair Professor, Hangzhou Normal University(youngm@hznu.edu.cn)

Bio
Youngmin Kim is currently Distinguished Research Professor Emeritus at Dongguk University, Jack Ma Chair Professor of Ma Yun Education Fund at Hangzhou Normal University,
Editor-in-Chief of Journal of English Language and Literature, Vice-President of International Association of Ethical Literary Criticism, Executive Council Member of International
Comparative Literature Association, Director of Digital Humanities Lab of Dongguk University. He is the founding Director of Institute of Trans Media World Literature, Dongguk
University. He was Director and Dean of International Relations, Dean of College of the
Humanities at Dongguk University, Visiting Professor at Cornell University and Sapporo
Gakuin University in Japan, Visiting Scholar and faculty affiliate of East Asia Center at the
University of Virginia at Charlottesville. He served as Advisory Committee Member of Institute of World Literature of Harvard University (2013-16), President of English Language andLiterature Association, Jacques Lacan and Contemporary Psychoanalysis Society, Yeats
Society of Korea, Vice President of International Association of the Study of Irish Literatures.
He is currently the Principal Investigator of “Trans Media, Digital Humanities, and World
Literature” (2017-2020) Project. He wrote articles and books on modern and contemporary
poetries in English, critical theory, transnationalism, translation studies, psychoanalysis,comparative literature and world literature, and Digital Humanities.

1

JADH2020

Plenary - (1)Abstract
Maryanne Wolf in her Proust and the Squid (2007) posits reading as a human invention, andelaborates the human brain’s plastic ability in relation to the act of reading: “Underlying
the brain’s ability to learn reading lies its protean capacity to make new connections among
structures and circuits originally devoted to other more basic brain processes that haveenjoyed a longer existence in human evolution, such as vision and spoken language. We
now know that groups of neurons create new connections and pathways among every time
we acquire a new skill” (5). Wolf’s understanding of the brain’s “plastic design” to “makenew connections among structures and circuits” is based upon the process of recollection
which is activated in the reading brain in milli-second. The human reading brain, designed
to store and retrieve words, can “elicit an entire history of myriad connections, associations,and long-stored emotions” in human evolution such as vision and spoken language. Wolf
claims that the two dimensions of the reading brain’s development and evolution are theintellectual and the biological, using French novelist Marcel Proust as metaphor for theintellectual and the squid as analogy for the biological. Proust saw reading as “a kind of
intellectual sanctuary” where human beings could provoke their intelligence and desires toexperience the Real out of their transformed imagination. Scientists in the 1950s used thesquid to illustrate “how neurons fire and transmit to each other, and in some cases to see
how neurons repair and compensate when something goes awry” (6).
The fact that the biological and cognitive function of the human brain in connecting and integrating at rapid-fire speeds without a single moment of consciousness brings the questionof the automation. The “unconscious” autonomy of the human agent is now confronted
with the automation which loses its human autonomy and transforms itself into the system
of automatism, and the matter is sublimated into the digital. Wolf’s models of Proust andthe Squid in terms of the intellectual and the biological is closely related to the linguistic andthe neurocognitive aspects of the Artificial Intelligence. The complementary examples ofhuman brain’s reading processes have analogically elaborated how various neuro-cognitive
processes will work algorithmically in the data-processing of the AI. Both cases of reading
by Wolf refer to human intelligence’s information processing in terms of the human brain’s
automatic learning, reminding us of machine learning and deep learning algorithms.
The development of the AI in tandem with that of human intelligence may be the last
great challenge of humanism and the first great endeavor of posthumanism. Cognitive
neuroscience and artificial intelligence have undergone revolutionary changes in the past
decades, and they foreground the embodied and environmentally embedded nature ofintelligent action. The augmentation and absorption of human agents by the digital now
seems inevitable, leaving the question of man and technology initiated by Heidegger still
incomplete. What is at stake is the ethical articulation of intelligence (both human and2

JADH2020

Plenary - (1)artificial) in this “second machine age.” One may recall Heidegger’s essay, “The Questioning Concerning the Technology,” in which two possible directions for ethical articulation
concerning technology are presented. On the one hand, Enframing (Gestell) challenges
forth into the frenziedness of ordering that blocks every view into the propriative eventof revealing and so radically endangers the relation to the essence of truth. On the other
hand, Enframing (Gestell) propriates for its part in the granting that lets man endure–as
yet inexperienced, but perhaps more experienced in the future–that he may be. the one
who is needed and used for the safekeeping of the essence of truth. Thus, the rising of thesaving power appears (314). Whichever route they may choose, humans are struggling
with the “existential threats” of facing posthumanism which initiates the retreat of thehuman agent into the background of a larger eco-technological environment. What is atstake is the ethical articulation of intelligence (both human and artificial), tools, machines,
and forms of life in this “second machine age,” described by Erik Brynjolfsson and Andrew
McAfee. If we turn to the other side of the coin, the new environment of the unlimited
possibilities of hyper-connectivity and convergence in this age of what Klaus Schwab called“The Fourth Wave of the Industrial Revolution” emerges, revealing “emerging technology
breakthroughs” across the physical, digital, and biological worlds: neural network structured artificial intelligence research, big data driven social media, the rapid adoption of 5G
small screen device computer technology, reality augmenting software, and what not.
In this context of posthumanism and second machine age, we are confronting the epochal
crisis. When we zoom out from the global scene of the pandemic coronavirus, humanbeings seem to be confronting an imagined threatening geological Anthropocene in thebiosphere. When the globe is zoomed in among humans even down to the cells, genes andchromosomes of the human body, we can reach the challenging disruptive transformationof our everyday lives, covering our inner lives, work, education, social activities. How tobalance between zooming-in and zooming-out properly is what is at stake in our daily lives.
The small touch screen of the multiple electronic devices we use in time will provide theplatforms for us to find, read, think, feel, care, and even survive.
This talk will attempt a poetics of scale in world literature in terms of the perspective,
method, and logic in dealing with the database of the world literature.

3

JADH2020

Plenary - (2)The Regional Turn in Digital Humanities in India
Decolonizing Digital Knowledge Infrastructures
Nirmala Menon1
1

School of Humanities and Social Sciences, Indian Institute of Technology
2(IIT)
Digital Humanities and Publishing Research Group, Indian Institute ofTechnology (IIT)

Bio
Nirmala Menon leads the Digital Humanities and Publishing Research Group at the Indian
Institute of Technology (IIT), Indore, India. She is a faculty member of the School of Humanities and Social Sciences (HSS), Discipline of English, IIT Indore. She is the author of Migrant
Identities of Creole Cosmopolitans: Transcultural Narratives of Contemporary Postcoloniality(Peter Lang Publishing, Germany, 2014) and Remapping the Postcolonial Canon: Remap,
Reimagine, Retranslate (Palgrave Macmillan, UK 2017). She has published in numerous
international journals and speaks, writes and publishes about postcolonial studies, digital
humanities and scholarly publishing. She directs Ph.D. students is in their projects and runs
DH projects from the research lab at IIT Indore. Her primary area of research is Postcolonial
Literature and Theory. Her focus is on the comparative study of twentieth century postcolonial literatures in English, Hindi and other languages. Digital Humanities, Gender studies,
Globalization and Translation studies are additional areas of research. Her interests are
multilingual but also interdisciplinary; her research examines the ways in which literaturesfrom different non-Western languages influence and can redefine and reframe postcolonial
theoretical concepts. She is also on the Advisory Board of the Open Library of Humanities(OLH) , Advisory Board member of Ubiquity Press, UK and Advisory Board Member, Open
Access India and Chair, (2016-17) CLCS Global South Forum, Modern Language Association(MLA), Prof. Menon is one the founder members and current President of Digital Humanities
Alliance in Research and Teaching Innovation (DHARTI)

4

JADH2020

Plenary - (2)Abstract
This talk will examine the emerging areas of research in Digital Humanities in India by an
examination of the term “Digital Humanities” and its uneven and uncertain trajectory in
the Indian research ecosystem. As with other systems around the world, DH in India iscomplex and contextual and each of us involved in the disciplinary practice encounter
resistance and skepticism. Resistance to what is perceived as a deliberate distancing
from the larger Humanities queries, skepticism about appropriating the rhetoric of STEM
disciplines merely as a tool for greater legitimacy and validation. But of course DH is
far more than a sum of its criticisms and skepticisms I will address why it is imperativethat the digital be harnessed, critiqued, deconstructed and dissected precisely so thatthe issues that Humanities scholars have been invested in continue to be at the centre of
the development, deployment and dissemination of technology. Using the example ofDecolonizing Knowledge Systems through Scholarly Publishing I underscore the need tomake knowledge productions multilingual and accessible to students and researchers in
different parts of the country and the role of Open Access India in making education accessa little less unequal than it is now. I will discuss IIT Indore’s Project KSHIP and some of thecontinuing challenges in launching the project.
Digital Humanities pedagogy in India is a journey–a data point that is part of a cosmic,
but networked chaos. As scholars who work in the intersectional identities of the digitaland humanities, those data points are marked with a lot of support, mentoring, learningand unlearning. A survey of university courses across various small and big universities(forthcoming by Shanmugapriya and Menon) suggest that many small universities are also
recognizing the need to introduce Digital Humanities courses as a primer for their undergraduate students. The ubiquity of computational resources and techniques as an essential
part of Humanities inquiry is increasingly recognized across liberal arts departments but
the willingness, resources and institutional support to translate these into pedagogicaltools and curriculum is still nascent and emerging. I consider some of the reasons for the
resistance to institutionalizing digital humanities within the larger university systems in
India and conclude that the causes are historical but also perhaps the embedded rigidity
and regimentation of university-curricula across disciplines. Digital Humanities, as definedby its purported inter- or transdisciplinary character, becomes a tough sell within Indian
university ecosystems, often constrained by their colonial origins and the limitations of a
postcolonial social imaginary. By highlighting key issues regarding digital affordances inpostcolonial contexts and exemplifying digital humanities projects that were presented at
the DHAI 2018 conference, along with representative digital interventions beyond the conference’s ambit, we conduct what we may term a ‘distant reading’ of the digital humanities
in India. In conclusion, I underline the need for a postcolonial digital humanity that is deeply
invested in cross-disciplinary exchanges and pedagogical innovation: one that interrogates

5

JADH2020

Plenary - (2)hegemonic assumptions about both “Postcolonialism” and “Digital Humanities.”

6

JADH2020

Panel

Tracing playful media practice: Challenges for
Theory, Methodology, and Infrastructure
Martin Roth (Panel Chair)1,2 , Kazufumi Fukuda1 , Zoltan Kacsuk2 , and Akito
Inoue1
1

Ritsumeikan University
2
Stuttgart Media University

1

Panel Outline

Media practice today is not confined to a specific medium, platform, topic or language. In
times of intense “media convergence” (Jenkins 2006), of the “meta-medium” computer(Manovich 2013), and of ubiquitous digital networks, digital culture appears to happen
everywhere at the same time. Videogames are streamed on Twitch.tv, with the support ofdonations by “patrons” on respective crowdfunding platforms, next to streams of the U.S.
congress, which are commented on in the same manner as game play scenes (see Figure 1
and 2).
Media practice appears more dispersed and transgressive of media boundaries, of nationalboundaries, and of language boundaries. Platforms can be regarded as attempts at providing clarity. At the same time, they restrain, channel and control information and userpractices (Stalder 2016; Itō [伊藤] 2016). In this sense, contemporary media practice isboth unbound and highly controlled. On this basis, one of the future challenges for digital
humanities is how to theorize, capture and analyze such media practice adequately. This isa challenge for media theory and methodology, and one that requires new research infrastructures and tools. Focusing on digital games and Japanese visual media, this panel aims
to discuss these challenges and suggest avenues for solving them. The panelists present
their own approaches to making sense of contemporary digital media culture, and of making use of various available technologies and tools from the field of data science. Paper 1
discusses the possibility of modeling the heterogeneous data resources and using linked
data technologies for making them accessible to human and computer-aided analyses. Paper 2 analyzes large sets of fan-created data for revisiting Hiroki Azuma’s assumptions about

7

JADH2020

Panel

Fig. 1: Iconic Scene from the videogame Dark Souls (FromSoftware, 2011) played on
Twitch.tv

Fig. 2: Washington Post congress live stream of Michael Cohen’s testimony from Feb. 27,
2019, on Twitch.tv

8

JADH2020

Panelthe creation of otaku characters in Japanese anime, manga and games. Paper 3 explores
possibilities for understanding language-based differences in game culture, comparing text
data from a wide range of game cultural data resources.
In combination, these contributions can hopefully present a crucial problem that contemporary media research is facing, and relate our search for solutions to the work of otherareas of digital humanities that face similar issues.

Bibliography
Itō, Mamoru. 2016. “Dejitaru Media Jidai Ni Okeru Genronkūkan: Rironteki Tankyū No
Taishō Toshite No Segyō, Jōdō, Jikan.” Journal of Mass Communication Studies, no. 89:
21–43.
Jenkins, Henry. 2006. Convergence Culture: Where Old and New Media Collide. New Yorkand London: New York University Press.
Manovich, Lev. 2013. Software Takes Command. New York: Bloomsbury.
Stalder, Felix. 2016. Kultur Der Digitalität. Frankfurt (Main): Suhrkamp.

2(Kazufumi Fukuda) For the Development of a Data
Model for Tracing “Play” with Video Games and theMedia Culture

In considering the current video game culture, it is hard to ignore the existence of otherrelated media. One of the great examples is the case of “Dark Souls 3” (FromSoftware, 2016).
Given the game’s difficulty, large game world, and the variety of gameplay it offers, arguably,
few players have never seen the YouTube video of the game. At the same time, players
will often open strategy books and browse strategy sites as well. Interestingly, this gamehas an online competitive play element. Therefore, some players may send informationand communicate with other players via SNS such as Twitter. Thus, it can be said thatthe current video game play experience exists among video games and related media.
These are topics that have been discussed with keywords such as media mix (Steinberg2012) and convergence culture (Jenkins 2008). We think that various resources can belinked by constructing a data model that defines multiple media via video game works.
Numerous instances of them already exist on the Internet, either as user-generated contents
or as metadata for materials held by institutions around the world. By connecting theseresources, it becomes possible to analyze a wide range of play traces occurring in video

9

JADH2020

Panelgames and related media. The purpose of this study is to discuss future directions forconstructing such a data model and a research platform based on it.
Data models for describing video game materials and works have been discussed by manyscholars since the beginning of the 2010s (i.e., McDounogh et al. 2010, Jett et al. 2016,
Author et al. 2018, Hoffmann 2019). Besides, some researchers (Aarseth 2014, Juul 2016)
examine possible ontologies to define game characteristics. However, the existing works
take different approaches. The former adopts a model construction approach that assumes
the creation of bibliographic data and its adaptation. On the other hand, the latter is a
discussion about the “character of the game” and the “mode of existence” based on theframework of the existing philosophy, and does not take data creation into consideration.
Based on this difference, it is clear that a future subject for game studies with data models
is the inclusive connection between a bibliographic data and the philosophical framework
the author mentioned above.
Prior work has also been focusing on creating metadata for an entity/relational data model.
These data are published in the online catalog ”RCGS Collection (Fig 1)” owned by Ritsumeikan Center for Game Studies. Moreover, the data were contributed to ”Media Arts
Database (Fig 2)” known as a comprehensive catalog service of media arts provided by the
Agency for Cultural Affairs. In the RCGS Collection, linked data are published as dump data.
In addition, SPARQL endpoints are provided, enabling data retrieval and analysis using aquery language. Also, this dataset gets some of the authoritative data of the work from
Wikidata (Author 2019). Wikidata is characterized in that it is connected to many externalsources. Thus, it can be also said that their meta data contribute to the connection withexternal related resources.
Finally, we would like to discuss the problems and methodologies necessary to build adata model to properly describe the game-centric media culture mentioned at the outset.
As a major premise, it is necessary to establish a scope in order to select resources to bemodeled, because there is a huge amount of media related to games. To take shape in our
research, the case study approach may be most effective, because it allows us to collectvarious materials related to the playing experience from a selected game work. By analyzingthese materials, we can extract requirements and develop models. It is preferable thatmultiple cases be selected, and that they cover as much “ground” of what we perceive as
game culture as possible.
In conjunction with the case studies, a more specific designation may occur when discussing
the relationship with the game. Therefore, it is also necessary to consider a model for
designating the part of the content that constitutes the game work itself.
In this paper, we suggest solutions for these challenges and point at further work to be

10

JADH2020

Panel

Fig. 1: RCGS Collection Top Page

Fig. 2: Media Arts Database Top Page

11

JADH2020

Panel

Fig. 3: Applying “DARK SOULS 3” on the metadata model

12

JADH2020

Paneldone in the future.

Bibliography
Aarseth, Espen. 2014. “Ontology.” In The Routledge Companion to Video Game Studies,edited by Mark J. P. Wolf and Bernard Perron, 683–95. Routledge.
Author. 2019.
Author et al. 2018.
Hoffmann, Tracy. 2019. “Developing a Mediated Vocabulary for Video Game Research.” In
Proceedings of the Doctoral Symposium on Research on Online Databases in History 2019,11p. http://ceur-ws.org/Vol-2532/paper8.pdf.
Jenkins, Henry, and Mark Deuze. 2008. “Convergence Culture.” Sage Publications Sage UK:
London, England.
Jett, Jacob, Simone Sacchi, Jin Ha Lee, and Rachel Ivy Clarke. 2016. “A Conceptual Modelfor Video Games and Interactive Media.” Journal of the Association for Information Scienceand Technology 67 (3): 505–17. https://doi.org/10.1002/asi.23409.
Juul, Jesper. 2016. “Sailing the Endless River of Games: The Case for Historical Design
Patterns.” In 1st International Joint Conference of DiGRA and FDG 2016. Dundee. http://www.jesperjuul.net/text/endlessriverofgames/.
McDonough, Jerome, Matthew Kirschenbaum, Doug Reside, Neil Fraistat, and Dennis Jerz.
2010. “Twisty Little Passages Almost All Alike: Applying the FRBR Model to a Classic Computer Game.” Digital Humanities Quarterly 4 (2). http://www.digitalhumanities.org/dhq/vo
l/4/2/000089/000089.html.
Steinberg, Mark. 2012. Anime’s Media Mix: Franchising Toys and Characters in Japan. Univof Minnesota Pr.

3(Zoltan Kacsuk) Revisiting “Otaku: Japan’s Database
Animals” twenty years on: Can we use fan createddatabases to test one of Hiroki Azuma’s central arguments?

Media fans have been compiling data on their favorite works long before the internet became
the most important channel for communication (Jenkins, 1992; Okada, 1996; Yoshimoto,13

JADH2020

Panel2009). Naturally, with the advent of the internet this process of collecting and cataloging
information by individual fans and enthusiast communities has only become easier andamplified in its scale. The level of detail afforded by these fan compiled data sources has
also not gone unnoticed in academic research (cf. Hills, 2002). Although various online
databases created by enthusiast communities have become the go to resource for checking
information on hard to find media texts and artifacts, their use for large-scale quantitative
research has yet to become widespread.
Harnessing the power of fan created databases for academic research is precisely theaim of the Japanese Visual Media Graph (JVMG) project, funded by the German Research
Foundation’s (Deutsche Forschungsgemeinschaft, DFG) e-Research Technologies program.
By learning from and building on the experiences and content produced and aggregated byenthusiast communities, the project aims to create a database and query tool primarily foracademic researchers working on Japanese visual media. Furthermore, the JVMG project
employs a linked data framework for processing the various data sources and for creating
links between them. This approach not only allows for a great level of flexibility in dataprocessing, but also facilitates the linking of the data to even further sources, especiallylinked open data. Although the project is still in its development phase it has alreadyproven to be a powerful resource for approaching both new and long-standing questions inresearch on Japanese anime, manga, light novels and video games. To demonstrate thepossibilities opened up by such an integrated database as well as the challenges entailed
in working with data compiled by enthusiast communities this presentation will focus on
examining one of Hiroki Azuma’s most well-known works through the lens of data on visualnovels (a unique kind of narrative centered video game genre) and other types of Japanesevisual media, such as anime and manga.
Hiroki Azuma’s Dōbutsu ka suru posutomodan: otaku kara mita Nihon shakai (Animalizing
postmodern: Japanese society as seen from otaku), published in 2001, has been one of
the most influential treatises on not only Japanese otaku, but also on the production andconsumption paradigm defining Japanese anime, manga, light novels and video games inlate modernity. The book’s impact on the discourse around otaku and the just enumerated
domains is truly international thanks in part to the English translation, which was publishedin 2009 as Otaku: Japan’s Database Animals. With almost twenty years since the original
publication in Japanese and more than ten years since the English translation was released,
the concepts and fra to the lack of these types of quantitative research approaches in theconcerned fields, however, the lack of available data that is both large and granular enoughto use for such an analysis has also not been conducive to the emergence of quantitative
examinations of this sort.
This presentation will thus both introduce the wider JVMG project, and provide an example

14

JADH2020

Panel

use case showcasing the JVMG database’s affordances and limitations by examining oneof the central points from Azuma’s above cited work – namely that “many of the otaku
characters created in recent years are connected to many characters across individual
works, rather than emerging from a single author or a work” (2009 [2001]: 49) – with thehelp of the database. The use case will serve to illustrate certain inherent limitations of
databases compiled by enthusiast communities, and will also highlight how the JVMG
database with its incorporation of multiple such databases can serve to help mitigate someof those shortcomings.

Bibliography
Azuma, H. (2009 [2001]). Otaku: Japan’s database animals (introduction and translation by
J. E. Abel & S. Kono). Minneapolis, MN: University Of Minnesota Press.
Hills, M. (2002). Fan Cultures. London: Routledge.
Itō, G. (2005). Tezuka izu deddo: Hirakareta manga hyōgenron e [Tezuka is Dead: Opening
up theoretical approaches to manga representation]. Tokyo: NTT Publishing.
Jenkins, H. (1992). Textual Poachers: Television Fans & Participatory Culture. London: Routledge.
Lamarre, T. (2009). The Anime Machine: A Media Theory of Animation. Minneapolis, MN:
University Of Minnesota Press.
Okada, T. (1996). Otakugaku nyūmon [Introduction to otakuology]. Tokyo: Ōta.
Suan, S. (2017). “Anime’s Performativity: Diversity through Conventionality in a Global
Media-Form.” animation: an interdisciplinary journal, 12(1): 62–79.
Yoshimoto, T. (2009). Otaku no kigen [The origin of otaku]. Tokyo: NTT.

4

(Akito Inoue) Can we measure cultural differences invideo games? Awareness of the problem from previous
research

The relation between human consciousness and ”language” has been a major point ofdispute in social and human science. In what sense can we say that human consciousness
also bound by the ”games” it plays?
As a human behavioral environment, ”playing video games” has several interesting characteristics. Compared to linguistic behavior, video games may be less affected by the15

JADH2020

Panel”vocabulary” or ”range of meaning” formed by a specific region. For example, if playing thesame game, both African and Japanese children may have similar experiences. There areprobably quite a few cases where the game is played in an identical way.
Based on this assumption, can we also say that the is game completely free from socialcultural differences? Most often, the answer can be “NO”, for the following two reasons.
Firstly, games are localized and the content slightly differs from country to country (O’Hagan,M., & Mangiron, C., 2013). Secondly, even when playing the same game, research (i.e.,
Feng.at al, 2007) shows that there are usually social differences between players that affect
the gameplay. For example, when men and women were asked to play the same game, adifference in gameplay tendencies was found in many studies (i.e., Linda Hughes, 1983,
Feng.at al, ibid).
Although there is not yet much research on differences in gameplay by region, the RPG
Studies suggest that there may be a significant difference in the strategies of raising the gamecharacter level (Endoh, 2019) based on different social environments. Cultural differencesin gameplay can be presumed to exist because of the following three approaches that havebeen applied to measuring cultural differences concerning games:(1) Differences in different versions of the same work (O’Hagan, M., & Mangiron, C., 2013)(2) Differences in gameplay in different play environments of the same work (PelletierGangon, 2019)(3) Differences in the same version and environment of the game (Feng.at al, 2007)
Based on the above, our study aims to further develop awareness of the problem of (3).
Particularly, we would like to clarify the cultural differences in games, especially thosefound in different language areas.

Research methods
In this study, we are gathering text data from game play walking through, strategy information about specific games, game preview releases, review, YouTube comments and othersources to see the differences between Japanese and English-speaking countries. On topof this, we are examining the blurring of language characteristics in categories of textual
data (strategy information, evaluation information, etc.).
As a preliminary result, we have found that the variation in the language used is less varied
for previews and strategy information. And we suspect that the variance in reviews andYouTube comments may be larger.
In relation to this prediction, we also suspect that the texts’ tendency describing behaviors

16

JADH2020

Panelstrongly linked to the video game play environment are less scattered than those describing
behaviors not strongly linked to the video game play environment.
In this presentation, we discuss the results of our pre-survey, and suggest next steps for the
analysis.

Bibliography
Linda Hughes (1983) ”Beyond the Rules of the Game : Why Are Rooie Rules nice?”, The Worldof Play: Proceedings of the 7th Annual Meeting of The Association for the AnthropologicalStudy of PlayFeng, J., Spence, I., & Pratt, J. (2007). Playing an action video game reduces gender Differences in spatial cognition. Psychological science, 18(10), 850-855.
O’Hagan, M., & Mangiron, C. (2013). Game localization. Amsterdam and Philadelphia: John
Benjamins Publishing Company.
ENDOH Masanobu, Koji MIKAMI (2019), Gamedo: A Face of Ludology to understand JapaneseGame Culture
Jérémie Pelletier-Gagnon (2019), Players, Cabinets, and the Space In-Between : Case Studies of Non-Ludic Negotiation of Video Game Cabinet Spaces in Japanese Game Centers,
Replaying 201917

JADH2020

Long Paper Session I - (1)Reviewing Digital Literature through Digital
Literature
David Thomas Henry Wright1
1

Nagoya University

Electronic or digital literature refers to literary works that are ‘native to the digital environment’ (Rettberg 2019: 6). In a simpler sense, electronic or digital literature is defined byits ‘unprintability’ and its reliance on computer code to exist. Additionally, Bell et al (2010)
pose that electronic literature can be defined as works that are ‘written for and read on acomputer screen that pursues its verbal, discursive and/or conceptual complexity throughthe digital medium, and would lose something of its aesthetic and semiotic function if itwere removed from that medium.’
On June 11, 2020, the inaugural issue of The Digital Review was released (it should be notedthat I, the author of this paper, am a co-editor of this publication). This journal, a sibling
publication to the electronic book review, is dedicated to the ‘preservation and publicationof innovative, born-digital essays’. This issue included two reviews of two separate worksof electronic literature: a review of The Pleasure of the Coast by J.R. Carpenter, titled The
Pleasure of the Infinite: An (In)finite Review of The Pleasure of the Coast by JR Carpenterby DTH Wright (myself); and a review of Blocked In by Anastasia Salter and John Murray,titled Playing the Hard Questions: A Twine Review of Blocked In by Anastasia Salter and John
Murray by Caleb Andrew Milligan. These reviews are not simply text reviews. Rather, they
are works that engage with the works’ digital formats, interfaces, and code as part of theircritique. In the same way filmmaker Jean-Luc Godard implemented the ‘essay film’, so too
do these works implement a born-digital critique.
J.R. Carpenter’s The Pleasure of the Coast (2019), offered in both English and French, appropriates, exaggerates, détournes, corrects, and corrupts text from Roland Barthes’ The
Pleasure of the Text (1973), Beautemps-Beaupré’s Introduction to the Practice of Nautical
Surveying and the Construction of Sea-Charts (1808), and Jean Giraudoux’s Suzanne et lePacific (1921). The work is presented in five sections: ‘the infinite coast’, ‘the technicalcoast’, ‘the incremental coast’, ‘the grammatical coast’, and ‘the route of La Recherche’.
The review of Carpenter’s work, The Pleasure of the Infinite, adopts the code used by Car-18

JADH2020

Long Paper Session I - (1)penter for the section ‘the infinite coast’. Within this scrolling, ‘infinite’, decentred form,
the review is positioned. In so doing, the review explores and reflects not only on the work,
but engages with the work on the its own terms.
Blocked In (2019) by Anastasia Salter and John Murray, explores in the language and artistic
value of video games by representing this ‘debate through the fusion of two competingaesthetic genres: the hypertextual Twine game, and the traditional arcade platformer.’
The work ends with a challenge to the reader to not only be an active participant in thework, but to create their own. In his review of Salter and Murray’s ‘game essay’, Milligan
also engages with Twine, a free open-source software for making interactive fiction. He
concludes that to fully appreciate and critique the work, one must engage with the formats
and interfaces.
Using these two digital reviews as examples, this paper proposes a model for reviewing andengaging with works of born-digital literature. Unlike typical reviews or papers, it posesthat it is not enough to merely reflect and theorise the work without also engaging with
its form and code makeup. This academic process is understood as a form of ‘practice-led
research’ as defined by Smith and Dean (2009). Additionally, the method of engaging withand remaking the code and form of the work in question is regarded as an example of ‘third
generation electronic literature’ as defined by Flores (2019). This paper also addresses thepotential problems with doing so: notably, copyright issues.

Works Cited
Bell, Alice, et al. 2010 ‘A [S]creed for Digital Fiction’, Electronic Book Review, 7 March, at
https://electronicbookreview.com/essay/a-screed-for-digital-fiction/ (accessed 6 July2020).
Carpenter, J.R. 2019 ‘The Pleasure of the Coast’, Luckysoap, at http://luckysoap.com/plea
surecoast/en/technicalcoast.html (accessed 6 July 2020).
Flores, Leonardo 2019 ‘Third Generation Electronic Literature’, Electronic Book Review, 7
April, at https://electronicbookreview.com/essay/third-generation-electronic-literature/(accessed 6 July 2020).
Milligan, Caleb Andrew 2020 ‘Playing the Hard Questions: A Twine Review of Blocked In by
Anastasia Salter and John Murray’, The Digital Review, 11 June, at https://thedigitalreview.com/issue00/hard-questions/index.html (accessed 6 July 2020).
Rettberg, Scott 2019 Electronic Literature, Cambridge: Polity Press.
Salter, Anastasia and Murray, John 2019 ‘Blocked In’, Hyperrhiz, 21, at http://hyperrhiz.io/h

19

JADH2020

Long Paper Session I - (1)yperrhiz21/games/5-blocked-in.html (accessed 6 July 2020).
Smith, Hazel and Dean, Roger 2009 Practice-led Research, Research-led Practice in theCreative Arts, Edinburgh: Edinburgh University Press.
Luers, Will [Ed] 2020 The Digital Review, at https://thedigitalreview.com/about.html(accessed 6 July 2020).
Wright, David Thomas Henry 2020 ‘The Pleasure of the Infinite: An (In)finite Review of The
Pleasure of the Coast by JR Carpenter’, The Digital Review, 11 June, at https://thedigitalrevi
ew.com/issue00/pleasure-of-the-infinite/index.html (accessed 6 July 2020).

20

JADH2020

Long Paper Session I - (2)“He triumphs; maybe, we shall stand alone”
Using Correspondence Analysis to Investigate Modal Adverbs in Tennyson’s
Poems
Iku Fujita1
1

Graduate School of Language and Culture, Osaka University

This research aims to investigate the characteristic use of modal adverbs in Alfred Tennyson’s works. A number of studies on Tennyson’s poems have paid critical attention tostylistic similarities between Tennyson and other poets (Jordan, 1988; Ricks, 1987; Shaw,
1976; Ricks, 1969; to name but a few). As Leech (1969: 13) states, “[L]iterary archaism isoften inspired by the wish to follow the model of a particular writer or school of writers of thepast,” it can readily be imagined that authors follow the way that remarkable forerunners
had written, namely, their styles. However, having stylistic similarities does not necessarilymean reflecting literary artistry/quality.
Currently, the digital humanities approach is contributing to literary studies. (e.g. Hori,
2019; Tabata, 2016; Mahlberg and McIntyre, 2011). Poetry is no exception and this process isnow bringing new, additional knowledge to previous poetry studies from different aspects(e.g. Tartakovsky and Shen, 2019; Nakao and Jimura, 2016).
The study identifies characteristic modal adverbs in Tennyson’s works in comparison to theAugustan, Romantic, and Victorian poetry composed by Pope, Byron, Keats, Shelley, Browning, Coleridge, Wordsworth, Swinburne, D. Rossetti, Arnold, and Southey. Words underinvestigation include belike, haply, maybe, mayhap, peradventure, perchance, perhaps, andpossibly as modal adverbs, which mainly express epistemic possibility or uncertainty (Quirk
et al., 1985; Greenbaum, 1969; Poutsma, 1928). The adoption of these poets is based on
Shaw (1976), which indicates that there are resemblances in the styles between Tennyson
and these authors. Four authors have been added to obtain more comprehensive results.
The frequencies of eight modal adverbs are listed below in Table 1.

21

JADH2020

Long Paper Session I - (2)Table 1. Frequencies of eight modal adverbs for the 12 authors1
Tennysonmaybe
mayhapbelikeperchance
haply
possibly
peradventure
perhaps

31.65
11.51
25.9
109.34
5.75
2.88
11.51
94.95

Pope
0
0
0
11.02
8.82
6.61
0
218.28Wordsworth

Coleridge

Southey0040.26
123.31158.54
5.03
7.55
213.9

0
0
49.51
158.42227.7300158.42

0
2.2
44.02
90.25
110.060
6.6
134.27

Byron
0
0
5.16
208.9
18.05
7.74
0
479.69Shelley
0
0
0
33.42
16.71
12.53
0
87.74

Keats
0
0
0
29.88
139.4500209.18

Browning
9.78
11.4
40.73
114.05
105.9
50.51
37.47
276.97

Arnold0000102.6700174.54

Rossetti
00
085.3
159.94
10.66
10.66
85.3

Swinburne
4.69
00
103.13168.76
0
7.03
28.13

Correspondence analysis (CA) is useful in visualizing hidden relationships between variables,
namely the authors and modal adverbs in this paper. As Tabata (2004: 112) demonstrates,
CA is an effective method, in the case when data contains low-frequency content words
as variables for stylistic investigation. Tabata (2005) is one such study that sheds light on
the advantages of CA: “CA is one of the techniques for data-reduction alongside PCA and
FA. Unlike PCA and FA, however, CA does not require the intervening steps of calculating acorrelation matrix or a covariance matrix, and can, therefore, process the data directly toobtain a solution” (Tabata, 2005: 68).
A result of CA shows Tennyson as an outliner in Fig 1. While the other 11 authors are locatedbetween -0.5 and 0.5 along the vertical axis, Tennyson locates alone at the bottom of theplot. In this plot, relative distance between data points reflects similarity or contrast.
The column plot indicates that the variables maybe and mayhap are outliners (Fig 2). Again,
their proximity explains the similarity of their frequency patterns between variables: thecloser data points are located each other, the greater similarity they have between them inthe frequency patterns of modal adverbs. Additionally, the radius of the circle indicates therelative frequency of the word. Although the circle of maybe and mayhap are not larger
than any other variables, these two adverbs deviate from the other six adverbs.
In addition to the result of CA, the Mann-Whitney’s U test (MW test) was conducted to findsignificant frequency differences between Tennyson and the control authors. The MW test
is a nonparametric method which tests the variance of the medians. The test needs two
independent groups as samples; therefore, in this study, Tennyson is set as the target group(64 works), and the others are set as the control group (926 works). Table 2 shows theresult of MW test. The MW test result shows that haply, mayhap, and maybe are statisticallysignificant (p-value < 0.01). The effect sizes are all quite small, and thus there is a room for
improvement, this paper will consider the reason(s) of why these two adverbs stand out inTennyson’s works. Given the result of CA and MW test, mayhap and maybe can be listed ascharacteristic modal adverbs in Tennyson.
1Frequencies are normalized per million words

22

JADH2020

Long Paper Session I - (2)Fig. 1: The row plot of CA (12 authors)

23

JADH2020

Long Paper Session I - (2)Fig. 2: The column plot of CA (eight modal adverbs)

24

JADH2020

Long Paper Session I - (2)Table 2. The result of MW test

Word
perhaps
haply*
possibly
perchance
belikeperadventure
mayhap*maybe*

p-value
Effect Size (r)
0.28
0.03
0.00
0.13
0.38
0.03
0.03
0.07
0.86
0.01
0.15
0.05
0.00
0.09
0.00
0.20

Control>Target
Control>Target
Control>Target
Control>Target
Control>Target*: p < 0.01
Turning our attention to each work of authors, mayhap appears in Browning, Southey andTennyson’s works. In Tennyson, however, mayhap is used only his Lincolnshire dialectal
poetry so that mayhap can make the sound of his dialect in his poetry.
The OED (online) records first quotation of maybe dated in 1400. It mentions that maybe
was popularly used by writers in the seventeenth century; but, was not a standard word
before the mid-nineteenth century (s.v. maybe, adv., n., and adj.). In the case of Tennyson,
the first maybe appeared in 1842. The duration of time when Tennyson started to use maybe
and the time when the OED refers to maybe is quite short. Among the 12 authors, only
three Victorian poets used maybe, and none of the Augustan nor Romantic poets used
the word. This is in keeping with the description of the OED. One of the motives of its
usage is presumably the aforementioned “literature archaism” (Leech, 1969: 13). Moreover,maybe and mayhap behave differently, compared to the other six modal adverbs. These
do not collocate with other modal auxiliaries such as may, might, or can. That could be arepercussion of authors’ lexical choices as it led to differences in their prosody.
This study limits its discussion to eight modal adverbs in Tennyson’s poems; however, we
can also see some of the poet’s stylistic features. This paper investigates the use of modal
adverbs through quantitative analysis, yet qualitative analysis will be essential in future
research.

25

JADH2020

Long Paper Session I - (2)Acknowledgement
The author would like to thank the anonymous reviewers for the constructive feedback.

References
Greenbaum, S. (1969). Studies in English Adverbial Usage. Florida: University of Miami
Press.
Hori, M. (2019). “Itsudatsu-shita Collocation-to Idiom: Dickens-no The Pickwick Papers-no
Baai (Deviation of Collocations and Idioms: In case of Dickens’s The Pickwick Papers).”
Gengo Bunseki-no Frontier, 2019. pp. 308-321.
Jordan, E. (1988). Alfred Tennyson. Cambridge: Cambridge University Press.
Leech, N. G. (1969). A Linguistic Guide to English Poetry. London and New York: Longman.
Mahlberg, M. and McIntyre, D. (2011). “A case for corpus stylistics: Ian Fleming’s Casino
Royale.” English Text Construction 4:2. pp. 204-227.
Nakao, Y. and Jimura, A. (2016). The Canterbury Tales-no Shahon-to Kanhon-ni okeru
Gengo-to Buntai-ni tsuite (The language and style of the manuscripts and printed books of
the Canterbury Tales.) Hori, Masahiro. (ed.) Tokyo: Hituzi Syobou. pp. 21-52.
Poutsma, H. (1928). A Grammar of Late Modern English. 2nd ed. Groningen: P. Noordhoff.
Quirk, R., Greenbaum, S., Leech, G. and Starvik, J. (1985). A Comprehensive Grammarof the English Language. London and New York: Longman.
Ricks, C. (1969). The Poems of Tennyson. (ed.) London: Longman.
_____. (1987). The Poems of Tennyson in Three Volumes. (ed.) London: Longman.
Shaw, W. D. (1976). Tennyson’s Style. Ithaca and London: Cornell University Press.
Tabata, T. (2004). “—ly fukushi-no Seikihinndo Kaiseki-niyoru Bunntai Shikibetu: Koresupondensu Bunnseki-to Syuseibunn Bunnseki -niyoru Hikaku Kennkyuu (Style Identification of -ly adverbs by the occurrence frequencies: Comparing the correspondence
analysis and principal component analysis.” Denshika Gengo Shiryou Bunseki Kenkyu, 2003.
pp. 97-114.
_____. (2005). “Stylistics of –ly adverbs in Dickens and Smollet: A quantitative study bymeans of correspondence” Denshika Gengo Shiryou Bunseki Kenkyu, 2004-2005. pp. 65-78.

26

JADH2020

Long Paper Session I - (2)_____. (2016). Kyoucho Sakuhin-ni okeru Dickens-no Buntai (The Style of Dickens in Cowritten
Works.) Hori, Masahiro. (ed.) Tokyo: Hituzi Syobou. pp. 53-71.
Tartakovsky, R. and Yehsyahu, S. (2019). “Meek as milk and large as logic: A corpus
study of the non-standard poetic simile.” Language and Literature, 2019, vol.28, No. 3.
pp. 203-220.
The Oxford English Dictionary (online): https://www.oed.com/ (the last access date:2020-9-28)

27

JADH2020

Long Paper Session I - (3)Linked-Potter: an example of ontology for the
study of the evolution of literature and reading
communities
Federico Pianzola1,2
1

University of Milano-Bicocca, Italy
2

Sogang University, South Korea

In the case of libraries and other book archives, ontologies and linked data are mostly
used for metadata describing the materiality of books or paratextual information. I present
here an example of how an ontology can be used for the distant reading of literary texts,
to study literary history, the cultural evolution of fiction, or as a selection mechanism toidentify themes of interest. I created a knowledge base using the tags of the fanfiction
website Archive of Our Own (AO3) (Organization for Transformative Works, 2009), which has
implemented an excellent system of tags management (Dalton, 2012; McCulloch, 2019).
When publishing on AO3, authors can specify tags for characters, relationships, and additional freeform tags for any use they may think of. Autocompleting typing suggest canonicalforms for the tags, so that uniformity is guaranteed across the all archive. Moreover, specialized volunteers, called “wranglers,” aggregate synonym tags: e.g. “harrypotter” and “Harry
Potter” (AO3 Admin, 2012). The goal of AO3 is to help readers find exactly the kind of stories
they are looking for, but researcher can exploit the well-maintained and accurate tags
database to draw insights about the history and evolution of a specific genre of literature(fanfiction) and its readership. In particular, freeform tags offer authors the possibility tomake explicit in the metadata any relevant aspect of the story, like a psychological trait ofa character (e.g. “Morally grey Harry Potter”), a narrative strategy (e.g. “point of view ofDraco”), a setting (e.g. “Diagon Alley”), a timeframe (e.g. “post first war with Voldemort”),
etc. A distant reading of fanfiction through the lens of tags has benefits that go beyond theunderstanding of a widespread – and growing – cultural phenomenon. Data driven insightsfrom research on AO3 can be used to formulate better hypotheses regarding the evolutionof other cultural systems – like literary classics or genre fiction – and to more strategicallyplan labour-intensive and time-consuming tasks like manual annotation of textual corpora.

28

JADH2020

Long Paper Session I - (3)A current limitation of AO3 is that, when tags are aggregated, the “canonical tag” is linked
to the synonym tag in the backend database but in the frontend only the user-generated
synonym tag is displayed. Therefore, readers can retrieve all stories linked to a canonicaltag thanks to the built-in search engine, but the link is lot when content is scraped from thewebsite. In order to benefit from AO3 tags aggregation, but also to improve it for research
purposes, I used the software Protégé (Musen, 2015) and the Web Ontology Language (OWL)
to replicate AO3-generated ontology for the Harry Potter fandom, adding further subclasses
and properties. These are the steps I followed:
1. create classes for the four main categories (FandomTag, CharacterTag, RelationshipTag, and FreeformTag) and relevant subclasses: LoveTag, FriendshipTag,
FreeformCharacterTag, FreeformRelationshipTag, FreeformPlotTag, FreeformPlaceTag, FreeformTimeTag;
2. copy all the tags from the main page of the fandom tag “Harry Potter - J. K. Rowling”(Anon, n.d.), create objects of the type owl:NamedIndividual for each of them, andassign them to the respective classes;
3. define which tags are considered canonical in the AO3 database;
4. copy the synonyms of every canonical tag from the tag’s page, e.g. for “Hermione
Granger” (Anon, n.d.); create objects of the type owl:NamedIndividual for each ofthem; link them to the respective canonical tag through the property owl:SameAs;
5. link CharacterTags to the RelationshipTags through the property “participatesIn” anddefine “hasParticipant” as the inverse property;
6. link FreeformTags to the CharacterTags and RelationshipTags through the property“isTagOf” and define “isTaggedAs” as the inverse property;
7. to complete the whole knowledge base, run the reasoner (Sirin et al., 2007) to resolve
coreferences and infer axioms.
An example of how data are linked is shown in Fig. 1.
The resulting knowledge base has around 33,000 individuals (tags) linked between them. It
is complete with respect to all the data retrievable scraping AO3 frontend website for the
Harry Potter fandom. Using the knowledge base with a dataset retrieved in March 2020
(217,772 stories), the number of synonyms tags found for each class are: 247 CharacterTags(31,785 occurrences, 3.3% of the total occurrences for this class), 317 FreeformTags (54,517
occurrences, 4.6% of the total), 48 for RelationshipTags (13,833 occurrences, 4.5% of thetotal). Using aggregated tags allows to perform more complete analysis on the selected
dataset.

29

JADH2020

Long Paper Session I - (3)Fig. 1: Example from the knowledge base of the Harry Potter fandom on AO330

JADH2020

Long Paper Session I - (3)Moreover, specific information regarding various topics can be extracted from the knowledge base and used to analyze the metadata of the stories published on AO3 or to findsubset of stories with specific features. For instance, we can aggregate all synonyms of
the FreeformTags associated to the CharacterTag “Harry Potter” and count the number ofstories written every year about a certain character version of the young wizard. The themesthat attracts the most interest are female versions of Harry, his profession after graduating(Auror), the last stage of mastery of wizardry reached in the official novels (Master of Death),and a “dark” version which drastically changes the plot of the official novels (Slytherin Harry
Potter) (Fig. 2).

Fig. 2: FreeformTags associated to the character “Harry Potter” in different years. The fourmost frequent tags are highlighted in yellow
A broader study taking advantage of the Harry Potter ontology is Pianzola et al. (2020),
which studied the cultural evolution and growing diversification of Harry Potter fanfiction.
Compared to other archives of online fiction, like fanfiction.net and wattpad.com, AO3 has
metadata that can be very useful for the study of literature in a digital age. However, since
their database structure is not publicly available, knowledge bases like Linked-Potter haveto be created, with the advantage that additional information can be classified to explore
specific aspect of plot or style.

References
AO3 Admin. (2012). The Past, Present, and Hopeful Future for Tags and Tag Wrangling on
the AO3. Archive of Our Own. https://archiveofourown.org/admin_posts/267 (accessed 25
February 2020).
Dalton, K. L. (2012). Searching the Archive of Our Own: The Usefulness of the Tagging

31

JADH2020

Long Paper Session I - (3)Structure. University of Wisconsin-Milwaukee. http://dc.uwm.edu/etd/26/ (accessed 24
June 2013).
Harry Potter - J. K. Rowling. Archive of Our Own. https://archiveofourown.org/tags/Harry%20Potter%20-%20J*d*%20K*d*%20Rowling (accessed 5 May 2020a).
Hermione Granger. Archive of Our Own. https://archiveofourown.org/tags/Hermione%20Gr
anger (accessed 5 May 2020b).
McCulloch, G. (2019). Fans Are Better Than Tech at Organizing Information Online | WIRED.
Wired. https://www.wired.com/story/archive-of-our-own-fans-better-than-techorganizing-information/ (accessed 25 February 2020).
Musen, M. A. (2015). The Protégé Project: A Look Back and a Look Forward. AI Matters,
1(4), pp. 4–12. 10.1145/2757001.2757003.
Organization for Transformative Works. (2009). Archive of Our Own. https://archiveofo
urown.org/ (accessed 5 May 2020).
Pianzola, F., Acerbi, A. and Rebora, S. (2020). Cultural Accumulation and Improvementin Online Fan Fiction. OSF Preprint. 10.31219/osf.io/4wjnm.
Sirin, E. et al. (2007). Pellet: A Practical OWL-DL Reasoner. Journal of Web Semantics, 5(2),pp. 51–3. 10.1016/j.websem.2007.03.004.

32

JADH2020

Long Paper Session II - (1)Multinomial Mixed-Effects Models and Linguistic
Variation
Competitions among Japanese Subject-Honorific Constructions
Akitaka Yamada1
1

1

Osaka University

Introduction

When examining linguistic variation, it is important to detect the factors affecting selection
tendencies. For example, selection tendencies may change per verb (a linguistic factor) or
per register (a social factor), while some are attributed to sentence-level factors, and otheridiosyncrasies are ascribed to no clear factors at all. Thus, when only personal introspection
is used, proofs concerning how far each of these many factors contributes to a selection
become less convincing and objective.
To separate lexical idiosyncrasies from general tendencies, this research promotes a corpus
study employing an underused statistical model: the Multinomial Mixed-Effects Model(MMEM) (Leshvina, 2016). Applying this model to variations among Japanese subjecthonorifics, this study shows how scrutinizing random effects reveals lexical idiosyncrasies
and contributes to the discussion in theoretical linguistics.

2

Background

Japanese has several subject-honorifics, as illustrated in (1), and how they differ in use is ofgreat concern.
(1) a. sensei-ga go- tootyaku-ni nat-ta.
teacher-NOM HON-arriving-ni become-PST
b. sensei-ga tootyaku-nasat-ta.
teacher-NOM arrive-hons-PST
c. sensei-ga go- tootyaku-nasat-ta.

33

JADH2020

Long Paper Session II - (1)teacher-NOM HON-arrive-hons-PST‘The teacher arrived.’
Previous studies have revealed that (HYP 1) a verb of one mora cannot be used with the
honorific prefix, and (HYP 2) a verb of Chinese origin favors the nasar-construction whencompared to the go…ni nar-construction. Our goal is to examine to what extent thesewell-accepted conclusions are supported by the real corpus data.

3

Data, Model, and Estimation

The data set is a sample from the BCCWJ. Among the 20,864 hits, we concentrate on verbs
used at least more than 24 times, because it is not easy to detect the distributional profile ofa verb with a lower frequency. Based on these 11,223 cases, the study builds the following
MMEM model by developing the model proposed in Yamada (2019a):

The outcome variable is the choice of subject-honorific constructions in (1); the go…ni nar
construction is set as the baseline category; 𝛾⃗ (2) and 𝛽 (⃗ 2) represent the construction in(1b), and 𝛾⃗ (3) and 𝛽 (⃗ 3) represent the construction in (1c). The considered predictors are as
follows:(3) Group-level fixed effects:
a. SUPPL: 1 when the main verb has a suppletive form; 0 otherwiseb. MORA: 1 when the main verb is formed by one mora; 0 otherwise
c. ORIGIN: 1 when the main verb is a word of Chinese origin; 0 otherwise(4) Population-level fixed effects:

34

JADH2020

Long Paper Session II - (1)a. IMPs : 1 when the sentence is a strong imperative; 0 otherwise
b. IMPw : 1 when the sentence is a weak imperative; 0 otherwisec. AUX: 1 when the subject-honorific marker appears on an auxiliary; 0 otherwise(5) Group-level random effects:
a. LEXEME: a grouping variable indicating the predicate to which the subjecthonorific marking is attached
b. REG: a grouping variable indicating the register from which the sentence istaken.
To avoid the problem of complete separation, we assume weakly informative priors N(0,3) for fixed effect parameters, and Half-t distribution for the variance of random effects
Half-t 4 (0, .5).
The Hamiltonian Markov Chain Monte Carlo algorithm is adopted to estimate the parametersof the Bayesian MMEM. Stan is employed on R, allowing allows us the No U-Turn Sampler.
After confirming the convergence in estimation (with the R-hats of the estimated parameters
being < 1.01), the posterior distribution is interpreted for each parameter.

4

Results(a) Priors(b) Posteriors

Fig. 1.
Figure 1 illustrates the priors (left panel) and the corresponding posterior distributions (rightpanel); the light and dark gray regions represent the 95% and the 66% credible intervals,

35

JADH2020

Long Paper Session II - (1)respectively. In general, the posterior means of the fixed parameters do not substantially(2)(3)
differ between 𝛾⃗ (2) and 𝛾⃗ (3) , or between 𝛽 (⃗ 2) and 𝛽 (⃗ 3) , except for 𝛾02 and 𝛾02 , in agreement(2)(3)
with the view of (HYP 1). The 𝛾01 and 𝛾01 are also in line with (HYP 2). The IMPs shows thelargest effect size; (1a) rarely takes the strong imperative form, while (1b/c) have no such
restrictions (Yamada, 2019b). The positive value for AUX means that the construction in (1a)is not as easily used as an auxiliary as those in (1b/c) (Yamada, 2019a).
(3)

(3)

uj

uj

13
15
10
(2)
uj

3

(2)

2018
23

14711
21

uj

17

19
mora = 1
12

origin = 1

(3)
uj

5

15

24
8
6

16
2
1
9

22

12
25

(2)

(3)
uj

4

13

(2)

uj

uj
3

14711
21

19

Extreme values(> 1.96*sd)

suppletion = 1
1: age ′raise′

8: iw ′say′

15: ne ′sleep′

22: tabe ′eat′

2: ar ′be′

9: kaw ′buy′

16: nom ′drink′

23: touben ′state′

3: de ′come out′

10: kekkon ′marry′

17: riyou ′utilize ′24: yasum ′take a rest ′4: hanas ′talk′

11: kur ′come′18: sinpai ′be worried ′25: yos ′avoid ′5: ide ′come out′

12: kure ′give′

19: siteki ′point out ′6: ik ′go′

13: kurou ′have difficulty ′20: soudan ′consult′

7: ir ′be′

14: mi ′see′

21: sur ′do′Fig. 2.
While the results of the fixed effects essentially coincide with the findings of previous literature, MMEMs also allow us to examine idiosyncrasies not attributed to the aforementioned
structural/general tendencies. Observe the scatterplots in Figure 2, which show how eachlexeme has its own lexical idiosyncrasy. The following findings are worth attention.
• Despite the general tendency, some Shino-verbs do not favor the construction in (1a)(kuroo and kekkon).
• Despite the general tendency, the verb de- can appear in the construction in (1a).

36

JADH2020

5Long Paper Session II - (1)Implications for Future Study

These findings show that the generalizations in (HYP 1) and (HYP 2) are not absolute rules,
and can be overwritten by lexical idiosyncrasies. Although due to space limitations, thisresearch refrains from providing an analysis, it will be important for future study to examine
why these “outliers” exist. The fixed effects findings also require further investigation. For
example, the interaction with an imperative is a legitimate concern, which would be better
discussed from pragmatic perspectives (Yamada, 2019b). The issue of auxiliary status canalso be approached via the grammaticalization theory. Whatever theoretical frameworkis adopted, analysis must be based on empirical facts. This topic cannot be approachedwithout data analysis, such as that completed in this study, which provides descriptive
desiderata and, thus, serves as a necessary departure point for theoretical investigations.

Acknowledgements
This work was supported by JSPS Grant-in-Aid for Young Scientists (Start-up) #20K21957,
and the NINJAL collaborative research project “Cross-linguistic Studies of Japanese Prosody
and Grammar.”

References
Levshina, N. (2016) “When variables align: a Bayesian multinomial mixed-effects model ofEnglish permissive constructions.” Cognitive Linguistics, 27(2): 235-268.
Yamada, A. (2019a). “Competing subject-honorific constructions with predicates of Yamato
origin [Wago kigen no doosi to kyoogoosuru sonkeigo koobun].” Proceeding of the 168th
Meeting of the Mathematical Linguistic Society of Japan, pp. 66-71.
Yamada, A. (2019b). “An OT-driven dynamic pragmatics: high-applicatives, subjecthonorific markers and imperatives in Japanese.” Proceedings of Logic and Engineering of
Natural Language Semantics 16, pp. 172-185.

37

JADH2020

Long Paper Session III - (1)Digital Editing of Japanese Buddhist Texts from theManuscript to TEI and Database Building
Gaétan Rappo1
1

Institute for Research in Humanities, Kyoto University

This presentation is a direct continuation of my previous work on n-gram frequency analysis applied to assessing the authorship of a series of medieval Japanese Buddhist texts,
published in the 2018 in the Japanese Journal of the Association for Digital Humanities. It
aims to develop better tools and workflows by tackling both the problem of digital editions,and their insertion into a working database.
This series of Buddhist texts—called shōgyō (“sacred teachings”)—was largely ignored byhistorians for a long time, but the situation is gradually changing. These texts were mostly
composed in Japanese kanbun (classical Chinese read using the Japanese word order), andtheir writing style is extremely fragmentary, almost cryptic at times. In fact, they are notdesigned to be comprehended by the non-initiated, and often require oral transmissionof knowledge from a master in order to be truly understood. In order to determine theircontents and to utilize them as proper historical sources, one has to reconstruct the vast
knowledge network upon which they were built. Previous work by both Japanese andWestern scholars has demonstrated that computer-assisted analysis of this massive corpus
of texts, spanning an extremely long period, serves as a valuable tool to accomplish thistask.
Building on this work, this presentation is divided into two main parts. The first, “edition,”starts with the manuscript (or in some cases printed editions), focusing on the actual processof typing the text and editing it into a workable format. I will mention a few encoding issues,
especially with rare Siddham characters. Great progress has been made in the creation ona unicode font, including ligatures and characters created in Japan, so this section will both
deal with temporary workarounds and more permanent solutions that will hopefully beattainable in the near future.
However, the main focus will be on what can be done with the text once it is digitized. Inorder to achieve a better understanding of this extremely nuanced genre of literature, a38

JADH2020

Long Paper Session III - (1)slightly custom TEI framework will be applied to the texts. In this context, I will expand
and apply to the TEI framework a system of speaking voices that I proposed in a previous
article (2018). The idea is to distinguish passages written by the author, quotes from either
previous masters or canonical texts, notes and commentary outside of the main text, andreading indications, in order to better isolate the most meaningful parts of a given text. By
tagging parts of the texts according to their level of discourse, my aim is to refine the data
and to easily divide quotations of previous sources from original content. This, in turn, will
help better define the originality of each text as well as its intertextual relationship to eachother.
The second step is a direct continuation of the first, and will propose a few ways to integratethese texts—and their particularities—within a working database. My work will be based on
my ongoing collaboration with Professor Christian Wittern, especially regarding his Kanseki
repository. I will provide a few examples whose implementation could pose problems, suchas the existence of multiple manuscripts, differences in the Japanese readings (kundoku)
found in each copy of the text, and assess a few possible solutions, taking into account thenext possibilities given by my editing workflow.
In the context of both creating a digital edition and implementing a database, the form of
the document itself, and especially the treatment of marginal indications (such as notesin the upper and lower parts of the page and characters written on the backside as wellas kundoku markers) will be crucial. These elements are closely related to the text, butthey were not necessarily generated by its author and may have been added at any point
during the transmission process of a given manuscript. To give a concrete example, there
are several different extant manuscripts of the Goyuigō daiji, a text first composed by the
Shingon monk Monkan (1278-1357) in 1328. Close inspection of the colophons, characters
written on the backside, and reading indications, show that there are at least three different
lineages of manuscripts of this text. Such variations are not limited to subtle discrepancies
in the contents. In fact, the presence of different reading indications can lead to a distinct
way of reading and understanding the text itself. This will lead me to stress the need for amanner to visualize such genealogies in a database, and question the ways we define whata text or a work actually is.
In my conclusion, I will also assess the value of my findings not only as tools, but also
within a broader philological context. My focus will be on the problem of the authorship ofthese texts, building on the pioneering book published by Raji C. Steineck and Christian
Schwermann. In fact, this presentation will allow me to question further the very notion ofwork in such composite, and even cumulative works, building on and expanding previous
research on the subject with new data.

39

JADH2020

Long Paper Session III - (2)Between óneiros and pragma: the social context ofvideo game humor
Artur Skweres1,21English department, Faculty of Pedagogy and Fine Arts in Kalisz, Poland
2
Adam Mickiewicz University in Poznan, Poland

Short bio:
Artur Skweres, PhD, is an Assistant Professor at the English Department of the Faculty ofPedagogy and Fine Arts, Adam Mickiewicz University in Poznań. His academic interests
include American culture (especially literature and film), as well as theories of comedy
and humor. In 2017 he published a monograph on the play principle in comedy titledHomo Ludens as a Comic Character in Selected American Films, followed by the monographs
McLuhan’s Galaxies: Science Fiction Film Aesthetics in Light of Marshall McLuhan’s Thought(2019) and The relationship between oneiric and pragmatic play in Mark Twain’s works (2019).

Abstract
Despite the apparent relation between play and comedy, relatively few publications havebeen written about the relation between video games and the concept of humor (e.g. see
Dormann and Biddle, 2009 or Kallio and Masoodian, 2019). To fill this apparent lack, thepaper will consider the use of humor and comedy in video games, with special attention
given to their social context, since games are inherently a social phenomenon. Although the
new medium of games borrows from the older media, incorporating many of their forms
and functions, it also allows for new applications of humor. The vast majority of video
games, although they can be classified as belonging to different genres, are commonly
centered around conflict. One of the strategies game designers employ in averting theperception of games as causing primarily negative emotions (such as hostility or aggression)is the use of humor. For instance, the developer of one of the games which are going to
be discussed, Toshihiro Nagoshi, argued that the humor underlying the deeply dramaticand serious narrative of his games is inserted for pragmatic reasons. Without the touch of
comedy, which he considers necessary, the storylines in his successful Yakuza series would

40

JADH2020

Long Paper Session III - (2)be imbalanced and lack variation (Espinelli, 2017). Comical elements, therefore, not only
serve to ease the tension of the more serious moments, but serve as one of the underlying
pillars of the game’s ludo-narrative construction, ensuring its proper balance, which would
meet the player’s expectations. Such strategies also common outside of games, in real-life
context. For instance, Maemura (2014: p. 104) analyzes how humor is used in negotiation
practices by Japanese people who are aware of the existence of the social concept of kuuki
空気, which signifies the atmosphere or the feeling one gets in the company of other people
at a given time. The examples given by Maemura (2014: p. 117) reveal that the conflictavoidance and desire to avert the possibility of hostility is resolved through the use ofhumor and laughter, but also through conforming to behavior that is expected, thereby
relieving the possible tensions.
Hence, it will argued that the use of humor in video games should be analyzed as part of alarger, social context. Building on the experiences from previous publications concerning
the use of ludic strategies in humor (Skweres, 2017 and Skweres, 2019), the paper will argue
that social context is an important common quality in both games and humor. Consequently,
the instances of comedic appeal should be reconsidered with the focus on the social context
of the player, the game environment (including the avatar, non-playable characters), as wellas the environment not directly related to the game (including the non-player audience).
Furthermore, expanding on the ludic categories of play and games proposed by Roger
Caillois, the paper will consider how the new categories of óneiros and pragma (introducedby the author in Skweres, 2017) lead to a deeper understanding of the attitudes of boththe player and his or her audience. Finally, since the focus of the paper is on the social
context, one of the examples of video game humor will consider the new genre of videogame
streaming in which the player acts as the comedian for members of his usual audience.

References
Dormann, C. and Biddle, R. (2009) “A Review of Humor for Computer Games: Play, Laughand More.” Simulation & Gaming, 2 August 2009
Espineli, M. (2017) “Yakuza Creator Talks 007 Inspirations, Wacky Humor, And What’s Next.”
https://www.gamespot.com/articles/yakuza-creator-talks-007-inspirations-wacky-humor/1100-6451146/
Maemura, Y. (2014) “Humor and laughter in Japanese groups: The kuuki of negotiations.”
Humor 2014; 27(1): 103–119
Oskari K. and Masood M. (2019). “Featuring comedy through ludonarrative elements ofvideo games.” Entertainment Computing 31 (2019). DOI: 10.1177/1046878109341390

41JADH2020

Long Paper Session III - (2)Skweres, A. (2017) Homo Ludens as a Comic Character in Selected American Films. Cham,
Switzerland: Springer.
Skweres, A. (2019) The relationship between oneiric and pragmatic play in Mark Twain’s
works. Cham, Switzerland: Springer.

42

JADH2020

Long Paper Session III - (3)Big Data as Archive: Data Centers and Digital Traces
as Future Historical Artifacts
Yennie Jun1 and Javier Cha1
1

Seoul National University

Future humanists will potentially have access to around 40 zettabytes (about 40 trillion
gigabytes) of data from 2020, data generated from smart devices, sensors, social media
platforms, and content delivery services (Reinsel, Gantz, and Rynding 2017). This immense
amount of information takes the form of unstructured, constantly changing, and highly
mutable activity streams, which are stored, processed, and cached in server farms, data
centers, and content delivery networks worldwide. What are the implications of the big
data paradigm to researchers and archivists?
The challenges of archiving mutable data sources are not new. Book historian Robert Darnton compares the fragmentary nature of early modern French “anecdotes” to present-day
blogging systems and social media platforms, claiming that “information comes in fragments and embeds itself in whatever niches are provided by the surrounding environment”(2013). Librarian David McKitterick explains that textual variance was common among
early printed books, within and without versions; “the mutable and malleable nature of theprinted word” is not unique to big data (Howsam 2006, 68).
On the other hand, big data is unlike anything seen in previous information regimes andmedia landscapes. Its sheer size and scale defy imagination. According to TechJury, “itwould take a person approximately 181 million years to download all the data from theinternet” (Petrov 2019). In 2019, 30,000 years of video content were uploaded to YouTube. A
typical data center with hundreds of thousands of stacked servers requires power stations
and backup generators nearby. In Ireland, 54 data centers, including Facebook’s Clonee
site, have access to 642 megawatts of power capacity, enough to serve 300,000 average
US households (Pollock 2019). In 2017, the US Library of Congress stopped their project ofarchiving Twitter because “conducting one search of the 2006 to 2010 tweet archive would
take 24 hours” (Wamsley 2017).
The social web’s drive to generate dynamic and highly personalized content makes it phys-

43

JADH2020

Long Paper Session III - (3)ically impossible to create a separate archive of what exists only in the specious present
that technologists call “real-time”. One of the main characteristics of Web 2.0 is its unprecedented pace of change. According to a report by the International Data Corporation (IDC),“consumers … expect to access products and services wherever they are, over whateverconnection they have, and on any device. They want data in the moment, on the go, andpersonalized” (Reinsel, Gantz, and Rynding 2017). Technologies such as personalization
platforms are designed specifically to provide unique experiences for each user. Social
media platforms and streaming services depend on “real-time” updates. Existing web
archives, such as the Internet Archive, are unable to do more than capture versions of static
pages and are not well equipped to deal with big data’s variability or volume.
Additionally, an increasing amount of this data is being stored in public or private cloudservices, rather than on personal hard drives. Therefore, we must consider post-custodial,
decentralized technological solutions and protocols rather than having a central location
for a digital archive. For example, solutions involving emulation, Universal Virtual Computer (UVC), and blockchain have been suggested, but each has their limitations (Lorie2014). This effort will involve immense technological and legal cooperation among national
governments, private companies, scholars, and many other actors.
This leads to the necessity of preserving not just the content, but the context and culture of
our current big data society. Merely preserving the raw data (content) cannot tell the stories
of how the data is used; capturing the diversity of user experiences (context) is crucial for
conveying a more complete picture of the Age of Big Data. Archiving the context requires
emulating the user experiences of interacting and creating data within the larger information ecosystems and determining what kind of surrogates would best represent theseexperiences. Director of Coalition of Networked Information (CNI) Clifford Lynch suggests
using robotic witnesses to create and capture fabricated experiences of different social
media platforms to document a broad set of experiences (Lynch 2017). Other solutions
include video-recording users’ physical interactions with technology, capturing the actual
output of the technologies, and creating a snapshot in time of the database.
There is no such thing as a perfect archive; historian Ian Milligan mentions that ”no archiveis a true reflection of the world” (Milligan 17). As with books (“even if their texts can comedown to us unchanged - a virtual impossibility ... our relation to those texts cannot be
the same as that of readers in the past”), we cannot perfectly and accurately capture theentirety of Web 2.0 (Darnton 1990). What, then, is the goal of big data as an archive? Lynchsuggests that it is to”preserve a reasonably accurate sense of the present for the future”(2017). I argue similarly, that such an archive must preserve, capture, and convey big
data in meaningful ways for future generations to best understand our present culture.
It is imperative that we preserve both the physical data and the user experiences to best44

JADH2020

Long Paper Session III - (3)represent our present-day relationships with big data.

Works Cited
Darnton, Robert. 1990. “First Steps Toward a History of Reading.” In The Kiss of Lamourette:
Reflections in Cultural History, 154–87. New York: W.W. Norton. http://robertdarnton.org/si
tes/default/files/First%20Steps%20Toward%20a%20History%20of%20Reading.pdf.
———. 2013. “Blogging, Now and Then (250 Years Ago).” European Romantic Review 24 (3):255–70. https://doi.org/10.1080/10509585.2013.789694.
Howsam, Leslie. 2006. Old Books and New Histories: An Orientation to Studies in Book andPrint Culture. Toronto: University of Toronto Press.
Lorie, Raymond. 2014. “Long Term Preservation of Digital Information.” IBM Almaden
Research Center. https://zoo.cs.yale.edu/classes/cs426/2014/bib/lorie01long.pdf.
Lynch, Clifford. 2017. “Stewardship in the Age of Abundance.” First Monday 22 (12–4).
https://firstmonday.org/ojs/index.php/fm/article/view/8097/6583.
Milligan, Ian. 2019. History in the Age of Abundance? How the Web Is Transforming Historical
Research. Montreal: McGill-Queen’s University Press.
Petrov, Christo. 2019. “Big Data Statistics 2020.” TechJury. https://techjury.net/statsabout/big-data-statistics/#gref.
Pollock, Sean. 2019. “Hundreds of Millions’ to Be Invested in Data Centre in Meath.” Independent.Ie. 2019. https://www.independent.ie/business/irish/hundreds-of-millions-to-beinvested-in-data-centre-in-meath-38720181.html.
Reinsel, David, John Gantz, and John Rynding. 2017. “The Digitization of the World From
Edge to Core.” International Data Corporation. https://www.seagate.com/files/wwwcontent/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf.
Wamsley, Laurel. 2017. “Library of Congress Will No Longer Archive Every Tweet.” NPR.
2017. https://www.npr.org/sections/thetwo-way/2017/12/26/573609499/library-ofcongress-will-no-longer-archive-every-tweet.

45

JADH2020

Short Paper - (1)Digital Public History during the COVID-19
Pandemic: Discussion on the Corona Archive @Kansai University
Nobuhiko Kikuchi1 , Keiichi Uchida1 , Tadakatsu Okada1 , Takefumi Hayashi1 ,Takao Fujita1 , Satoshi Ninomiya1 , and So Miyagawa1
1

1

Kansai University

Purpose of the Study

Toward the end of 2019, the coronavirus disease (COVID-19) quickly spread from China to
the world. As of May 2020, Japan is under a state of emergency and, seemingly, undergoing acomplete change from how it was before the COVID-19 outbreak. In this crisis situation, oneof the aims of public history is to enable the recording of the rapid worldwide change thatis being experienced by people and pass on memories of life under pandemic conditions to
the next generation.
This study discusses the Corona Archive @ Kansai University [Figure 1], which was initiatedon April 17, 2020 in response to the COVID-19 pandemic by the Kansai University Open
Research Center for Asian Studies (KU-ORCAS), from the digital public historical perspective.

2

Significance of the Study

The Corona Archive @ Kansai University is a community-based digital archive project thatcollects records and memories from the students, faculties, staffs, and their families of the
Kansai University during the COVID-19 pandemic. The International Federation for Public
History (IFPH) is mapping the COVID-19 archival activities taking place in various countries
and, according to the IFPH, 308 cases have been registered as of May 1, 2020 (Cauvin, 2020).
However, there are few examples of such an initiative in Japan other than the Corona
Archive @ Kansai University. Hence, the significance of this study is that the Corona Archive@ Kansai University can be a model case for the community archival projects for pandemic
in Japan.

46

JADH2020

Short Paper - (1)Fig. 1: Screenshot of the Corona Archive @ Kansai University

3

Construction of the Corona Archive @ Kansai University

In this study, we particularly examined the following four topics associated with the Corona
Archive @ Kansai University construction.
(1) Eligibility for submission
Targeting the entire nation or all the people living in Osaka Prefecture, where our university
is located, does not conform to KU-ORCAS’s aim of branding Kansai University’s research.
In addition, we are concerned that the management of increasing numbers of posts will bedifficult from the server specification and manpower perspective. Hence, we decided toallow submissions from Kansai University members alone.
(2) System construction and material collection
To ensure that even a single person could easily and quickly build a system that enables
users to register materials by themselves, we adapted the open source software Omeka
Classic. Using some plugins, the Corona Archive @ Kansai University enables users toregister text data, images, videos, and audio files and maps these materials on the Open47

JADH2020

Short Paper - (1)Street Map. In addition, the metadata conform to the DCMI Metadata Terms.
(3) Treatment of rights
For data release, we have requested contributors in advance to release their data under the
CC BY-NC 4.0. In addition, we have considered the portrait and privacy rights of individuals
depicted in photographs. Therefore, the Corona Archive @ Kansai University asks users to
confirm the license and whether they have received permissions from the photographedindividuals to archive the data. Further, before publishing the materials, we check the data
to ensure that they do not violate the privacy of individuals. The users themselves can
indicate their intention to keep their posted data private.
(4) Long-term storage
KU-ORCAS is a time-limited project, and the long-term preservation of the collected datacannot be carried out by KU-ORCAS alone. Accordingly, we have developed a data preservation system in cooperation with the university museum and archive.

4

Situation after the Project Implementation

As of May 1, 2020, at the time of writing this paper, it is about two weeks since the start of the
project; hence, I can describe only the situation immediately after project implementation.
Two weeks have passed since I started, so I can only describe the situation immediately
after the start.
As of May 1, there have been 22 registrations. Among them, two were kept private: one casewas set as private by the contributor himself, and the other was made private because it
contained sensitive personal information.
As described earlier, any members of Kansai University can contribute to our digital archive,
although the contributors are almost always only KU-ORCAS members. Therefore, most of
the posts are centered on the university’s Senriyama Campus (Suita City, Osaka Prefecture),
where KU-ORCAS is located, although Kansai University has several campuses. In addition
to text data, images, videos, and audio files can be attached to posts; however, currently,
only image files are accepted as attachments to posts.

5

Future Issues

There are many challenges ahead. Many challenges exist in the maintenance of the Corona
Archive @ Kansai University. In particular, significant issues are involved in material collection. The KU-ORCAS does not have a high level of visibility on campus because it is a

48

JADH2020

Short Paper - (1)time-limited organization. In addition, because of the restrictions imposed in responseto the COVID-19 outbreak, the university’s public relations tools (official Twitter and Facebook accounts, and press releases provided by the university’s Public Relations Division)
were allowed to consider only topics related to educational affairs. In addition, due to thewidespread need for social distancing, it is impossible to hold events that bring people
together. Therefore, requests to contribute records and memories have been limited to the
SNS and website of the KU-ORCAS, which further limit the collection of stories.
Another challenge pertains to the use of the collected data. The records and memories
of Kansai University members should be available for use by not only the university’s
constituents themselves but also future researchers. Hence, it is necessary to promote theconstruction of similar COVID-19 archives in Japan and strengthen the ties among domestic
and international archives.

Bibliography
Corona Archive @ Kansai University. https://www.annex.ku-orcas.kansai-u.ac.jp/covid19a
rchive, accessed 2020-09-22.
Kikuchi, Nobuhiko. “Branding East Asian Cultural Studies By”Opening” Access to Research
Resources, Research Groups, and Know-Hows”. DH2019. https://dev.clariah.nl/files/dh201
9/boa/0182.html, accessed 2020-09-22.
Cauvin, Thomas. “Mapping Public History Projects about COVID 19.” International Federation for Public History. https://ifph.hypotheses.org/3225, accessed 2020-09-22.

49

JADH2020

Short Paper - (2)Re-interpreting the Paintings and Poems of the
Lotus Sutra through a Co-occurrence Network
Religion and Life of the Aristocratic during the Japanese Medieval Ages
Aiko Aida1
1

Japan Society for the Promotion of Science (JSPS)The Lotus Sutra has for long been regarded as the most popular scripture in the Tripitaka,which was followed by Japanese emperors and aristocrats from the eleventh to fourteenth
centuries. (1). The most important version of the Lotus Sutra contains twenty-eight chaptersin eight fascicles that were copied as eight or twenty-eight scrolls by the nobles at the time.
According to one study (2), there exist approximately five hundred handscroll copies madebetween the eleventh and fourteenth centuries, which contain the frontispieces of the Lotus
Sutra (Sanskrit-Saddharma Pundarika Sutra). Some of these frontispieces are depicted ingold and silver inks on indigo-dyed papers, while others are drawn by richly colored mineralpaints on scattered gold or silver foils over papers. It is said that the latter were sometimes
based on Waka poems of the Lotus Sutra, called Hokekyo-ka (Japanese-法華経歌).
Thus, this paper intends to re-interpret the correlation between Waka poems or the Buddhist
paintings and the chapter’s themes of the Lotus Sutra in Medieval Japan, as derived from a
co-occurrence network between the two; this was achieved by using extracted vocabularies
of Waka poems by applying KH Coder, a free software for quantitative content analysis (3).
The KH Coder was utilized by applying MeCab (4), a morpheme analysis system, and 中古
和文 Unidic (5), an electrical dictionary for morphological analysis of classical Japanese.
Furthermore, in this process, about two hundred and thirty Waka poems were analyzed,
which could be identified as ten chapters each other (6). It is said that those Waka poems
might be the source of the paintings on the back of Chapter 11 of the Heike Nōkyō scroll
donated to the Itsukushima shrine around 1164.
The paper attempts to reveal that, for this research, motifs or symbols were extracted notfrom the Lotus Sutra but from the Waka poems based on it. This research also reveals
that most of the extracted words could be correlated with the chapters of the Lotus Sutra,
although there are common motifs in many of its chapters, such as Buddha, father, children,

50

JADH2020

Short Paper - (2)the moon, law, heart, and mountains. For example, Hana (花: flowers or cherry blossoms)
relates to Chapter 1, Ushi (牛／憂し: cow or distressed) to Chapter 3, Mi (実／身: nut orbody) to Chapter 12, Ama (尼／雨: nun or rain) to Chapter 13, Sumu (住む／澄む: live or
clean) to Chapter 16, Uki (浮き／憂き: float or unhappy) to Chapter 27, and Yama-no-ha(山の端: ridgeline) to Chapter 28.

Fig. 1: A Co-occurrence Network in the Hokekyo-ka
Moreover, the paper points out that the above keywords are essentially based on the Lotus
Sutra, and that some demonstrate paronomasia as pivot words (Japanese-掛詞). The
number of pivot words extracted from the Lotus Sutra was larger than expected. Thispossibly indicates that the extracted words in Hokekyo-ka poetries developed not with thecolored paintings on decorated papers, but rather with the Lotus Sutra paintings in goldand silver inks on indigo-dyed papers that were made under the Tang and Song dynasties
in China.

Bibliography(1) W. J. Tanabe, Paintings of the Lotus Sutra, John Weatherhill New York and Tokyo, 1988.

51

JADH2020

Short Paper - (2)(2) Nara National Museum, Lotus Sutra -Its Calligraphy and Decotation-, Nara National
Museum, 1987.
(3) http://khcoder.net/en/(4) https://taku910.github.io/mecab/(5) Toshinobu Ogiso, Mamoru Komachi, Yasuharu Den and Yuji Matsumoto. UniDic for
Early Middle Japanese: a Dictionary for Morphological Analysis of Classical Japanese, In
Proceedings of the 8th International Conference on Language Resources and Evaluation(LREC 2012), pp.911-915 (2012). https://unidic.ninjal.ac.jp/download_all/#unidic_wabun(6) Aiko Hashimura (Aida), The ideological and decorative program of Heike Nokyo: on therelationship between the images of the four seasons on the back of the ”Jeweled Stupa”
scroll and the pictorial schema for the twenty-eight chapters of the Lotus Sutra, Bijutsushi:a Journal of the Japan Art History Society (166), 2009-3. A. Hashimura, The Lotus Sutra in
the Kagawa Museum: an extant example of Japanese painting in 12th century East Asia,
The Kokka: a monthly journal of oriental art (1377), 2010-7.

52

JADH2020

Short Paper - (3)Computer Vision-based Comparison of
Woodblock-printed Books and its Application to
Japanese Pre-modern Text, Bukan
Thomas Leyh1,3 and Asanobu Kitamoto2,3
1
2

1

University of Freiburg

ROIS-DS Center for Open Data in the Humanities
3
National Institute of Informatics

Introduction

Digital Humanities have potential to reduce the complexity of bibliographical study by developing technical tools to support comparison of books. So far, the effort has been mainly
put into text-based methods on large corpora of literary text. However, the emergence oflarge image datasets, along with the recent progress in computer vision, opens up new
possibilities for bibliographical study without text transcriptions. This paper describes thetechnical development of such an approach to Japanese pre-modern text, and in particular
to Bukan, which is a special type of Japanese woodblock-printed book.
The authors have been approaching this problem in the past (KITAMOTO et al. 2018). They
proposed the concept of “differential reading” for visual comparison. Furthermore, (Invernizzi 2019) proposed “visual named-entity recognition” for identifying family crests,
using them for a page-by-page matching across different versions. This paper is a follow-up
of these works and proposes a keypoint-based method for the page-by-page matching,additionally yielding an option for highlighting differences.

2

DatasetThis work is mainly concerned with extracting information from a specific type of book: 武鑑—Bukan. These are historic Japanese books from the Edo period (1603-1868). Serving as
unofficial directories of people in Tokugawa Bakufu (the ruling government in Japan), they53

JADH2020

Short Paper - (3)include a wealth of information about regional characteristics such as persons, families andother key factors. See figure 1 for an example. These books were created with woodblockprinting. Because the same woodblock has been reused for many versions of the book—sometimes with minor modifications—visual comparison can reveal which part of thewoodblock was modified or has degraded.

Fig. 1: Shūgyoku Bukan (袖玉武鑑) from 1867, page 6; showing names, descriptions, family
crests and procession items. Especially interesting are the blank areas on the right, because
in other edition they contain text.
ROIS-DS Center for Open Data in the Humanities and the National Institute of Japanese
Literature are offering 381 of these Bukan as open data (Center for Open Data in the Humanities 2018). The original images have a width of 5616 and height of 3744 pixels. From theopen data we created a derived dataset using the following preprocessing methods. Under
the assumption that this task (1) does not require this level of detail, (2) does not require
information about color and (3) only compares the actual pages, not the surrounding area,
all scans are resized by 25%, converted to grayscale and finally cropped, resulting in animage shape of 990 × 660 pixels. If there are two book pages per scan, they are split attheir horizontal center, yielding a shape of 495 × 660 pixels per page.

3

MethodUsing an approach based on Computer Vision, two techniques were applied:
1. Keypoint Detection and Matching for finding the same features in different images.
2. Projective Transformations for comparing two different images regardless of their
original orientation.
We used the OpenCV software library (Bradski 2000).
Fig. 2: Evaluation of the performance of four keypoint detection algorithms. AKAZE UPRIGHT performs best.

54

JADH2020

Short Paper - (3)(a) F1 score over a range of threshold values, which is the harmonic mean of precision andrecall, thus peaking where both underlying metrics are high.

(b) Precision and recall, assuming an optimal threshold 𝑡 was chosen. This is an upperbound on the performance of keypoint detection for this dataset.

55

JADH2020

3.1

Short Paper - (3)Keypoint Matching

Keypoint Detection (Szeliski 2010, Ch.4) is about finding points of interest in an image
that are most noticeable and give a unique description of the local area surrounding them.
Computer Vision research produced various kinds of keypoints, most prominently SIFT
(Lowe 2004). For evaluating the performance of these algorithms, 12 prints of the Shūchin
Bukan (袖珍武鑑) were manually annotated, in total around 1800 pages, holding information
about pairs of matching pages.
Using these annotations, six keypoint algorithms were empirically evaluated1 by trying tomatch over all possible page combinations. This produces a list of similar keypoint pairs foreach combination. The number of these pairs is interpreted as score for the similarity of two
pages. Two pages are matched if the score is above a given threshold. On the one hand, a
low threshold results in more matches (higher recall), but a larger part is judged incorrectly.
On the other hand, a high threshold yields a larger percentage of correct matches (higher
precision), misclassifying actual matches at the same time. Using the annotations, precisionand recall were calculated for a range of thresholds. At this point, AKAZE UPRIGHT has thebest performance with both metrics around 0.8 for an optimal threshold value. See figure 2
for further details.

3.2

Projective Transformations

Projective Transformation (or Homography) for images is defined as a matrix 𝐻 ∈ 𝑅3×3
for transforming homogeneous coordinates 𝐻𝑥⃗ = 𝑦 ⃗ (𝑥⃗ and 𝑦 ⃗ are interpolated pixel coordinates). This operation results in linear transformations like translation and rotation, butalso changes in perspective (Marschner and Shirley 2015). For finding such a transformationfrom matching keypoints, the heuristic Random Sample Consensus (RANSAC) algorithm iscommonly used (Fischler and Bolles 1981). Basically, a random subset of matching keypoints is chosen, using this to compute a transformation and calculating an error metric. By
iteratively using different random subsets, eventually the transformation with the smallest
error is picked.
The benefit is twofold: First, the algorithm implicitly uses spatial information of the matching image candidates to filter out false positives, thus greatly boosting the matching performance. Precision and recall are close to their maximum of 1.0, see figure 3. Secondly,
it directly yields the transformation matrix 𝐻, enabling the creation of image overlays forvisualizing the differences. Looking at the perspective components of 𝐻, additional filtering
is done2 , removing even more of the remaining false positives.
1

ORB (Rublee et al. 2011), AKAZE (Alcantarilla and Solutions 2011), AKAZE without rotational invariance(UPRIGHT), BRISK (Leutenegger, Chli, and Siegwart 2011), SIFT, SURF (Bay, Tuytelaars, and Van Gool 2006)
»
»
»
»
2
By asserting that »»»𝐻3,1 »»» ≤ 0.001 and »»»𝐻3,2 »»» ≤ 0.001.
»
»
»
»

56

JADH2020

Short Paper - (3)Fig. 3: Applying RANSAC on the keypoint matches from AKAZE UPRIGHT results in better
matching results: The F1 score—see figure 2 (a)—is above 0.95 for a wide range of thresholds
up to 80, indicated by the grey horizontal line.

4

Discussion

This two-step-pipeline archives high precision and recall when looking for similar pages ofdifferent book editions. Performance seems to be robust with respect to most parameters.
For matching two actual pages, the number of matching keypoints is important since itis acting as a score. Storing this value allows a ranking of page pairs by visual similarity.
Furthermore, a threshold can be set dynamically, even after the computation of keypoint
pairs. Assuming semi-automatic application with a human assessing the results, recall
is of higher importance, thus a low threshold of around 40 is recommended, but can be
adjusted any time. See again figure 3 for the slope of the recall curve.
This was the basis for building a web-based prototype of a comparison browser: A scholar
in the humanities can browse a book while getting information about similar pages. Differences between pages can be visualized at any time, similar to figure 4. As a next step, weare designing a web-based tool that is easy to use and works for IIIF images.
Prof. Kumiko Fujizane, who is a Japanese humanities scholar specialized in this particular
type of books and collaborates with us, says that the major strength of our method is in
identifying differences which are hard to discern by human eye, while its weakness is inerrors caused by the distortion of pages by the book binding. Her request for future work is
to identify region-based difference in addition to page-based difference.

57

JADH2020

Short Paper - (3)Fig. 4: Visualizing page differences between two prints of the Shūchin Bukan (袖珍武鑑)from 1867. Differences are indicated by blueish and reddish coloring.

5

Conclusion

The proposed method has wide applicability in woodblock-printed books, because themethod uses only visual characteristics obtained from computer vision, and does not depend on the content nor the text of books. Although image alignment is a mature researcharea we also see recent developments, such as RANSAC-Flow (Shen et al. 2020), with potential to improve our results.

References
Alcantarilla, Pablo F, and T Solutions. 2011. “Fast Explicit Diffusion for Accelerated Featuresin Nonlinear Scale Spaces.” IEEE Trans. Patt. Anal. Mach. Intell 34 (7): 1281–98. https://doi.org/10.5244/C.27.13.
Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. 2006. “Surf: Speeded up Robust Features.”
In European Conference on Computer Vision, 404–17. Springer. https://doi.org/10.1007/11
744023_32.
Bradski, G. 2000. “The OpenCV Library.” Dr. Dobb’s Journal of Software Tools.
Center for Open Data in the Humanities. 2018. “武鑑全集とは？.” July 28, 2018. http://codh.rois.ac.jp/bukan/about/.

58

JADH2020

Short Paper - (3)Fischler, Martin A, and Robert C Bolles. 1981. “Random Sample Consensus: A Paradigm for
Model Fitting with Applications to Image Analysis and Automated Cartography.” Communications of the ACM 24 (6): 381–95. https://doi.org/10.1145/358669.358692.
Invernizzi, Hakim. 2019. “An Iconography-Based Approach to Named Entities Indexing in
Digitized Book Collections.” Master’s thesis, École polytechnique fédérale de Lausanne.
KITAMOTO, Asanobu, Hiroshi HORII, Misato HORII, Chikahiko SUZUKI, Kazuaki YAMAMOTO,
and Kumiko FUJIZANE. 2018. “Differential Reading by Image-Based Change Detection
and Prospect for Human-Machine Collaboration for Differential Transcription.” In Digital
Humanities 2018.
Leutenegger, Stefan, Margarita Chli, and Roland Y Siegwart. 2011. “BRISK: Binary Robust
Invariant Scalable Keypoints.” In 2011 International Conference on Computer Vision, 2548–55.
Ieee. https://doi.org/10.1109/ICCV.2011.6126542.
Lowe, David G. 2004. “Distinctive Image Features from Scale-Invariant Keypoints.” International Journal of Computer Vision 60 (2): 91–110. https://doi.org/10.1023/B:
VISI.0000029664.99615.94.
Marschner, Steve, and Peter Shirley. 2015. Fundamentals of Computer Graphics. CRC Press.
Rublee, E., V. Rabaud, K. Konolige, and G. Bradski. 2011. “ORB: An Efficient Alternative toSift or Surf.” In 2011 International Conference on Computer Vision, 2564–71. https://doi.org/
10.1109/ICCV.2011.6126544.
Shen, Xi, Francois Darmon, Alexei A Efros, and Mathieu Aubry. 2020. “RANSAC-Flow: Generic
Two-Stage Image Alignment.” In ArXiv.
Szeliski, R. 2010. Computer Vision: Algorithms and Applications. Texts in Computer Science.
Springer London.

59

JADH2020

Short Paper - (4)Clarification of boundaries and criteria for
periodisation in Beethoven’s career
Daisuke Miki1 and Akihiro Kawase111Faculty of Culture and Information Science, Doshisha University

Introduction

The compositional style of Ludwig van Beethoven (1770-1827) has conventionally been divided into three periods. According to Lenz (1855), who divided Beethoven’s compositional
style for the first time, the first, second, and third periods are respectively the periods thatspan from 1794 to 1800, 1801 to 1814, and 1814 to 1827. Several other musicologists, such
as Bücken (1931), Schenk (1967), Moroi (1983), and Hirano (2012), have proposed different
divisions. However, past musicologists do not share a common understanding of how to
divide Beethoven’s compositional style.
This situation could cause confusion about the stylistic evolution of Beethoven’s music. The
divisions in Beethoven’s compositional style were not originally made from the viewpointof musical theory, but instead based on major turning points in his biography. Therefore,
recent researchers have conducted quantitative analyses to identify the specific featuresthat characterise each period.
Richard (2011) found that the differences between the three periods of Beethoven’s works
can be explained by the contrast of intensity in dynamics, tempo, and harmony. Furthermore, Honingh and Bod (2011) revealed that the tritone interval category (IC) is an effective
feature by which to perform discriminant analysis to determine the periods in Beethoven’s
works. However, it is still unclear what specific types of chords and harmonic progressions
are unique in each period.
This study aims to clarify the periodisation boundaries in his compositional style anddetermine the specific features that distinguish one period from another. In this study, weapplied supervised learning to the relative frequency of N-gram extracted from Beethoven’s
all-string quartets (16 pieces; 70 movements) in terms of harmony, melody, and rhythm.
We chose the string quartets as our sample because Beethoven composed in this genre

60

JADH2020

Short Paper - (4)throughout his career.

2

Methodology

We conducted a discriminant analysis using decision trees and random forests to build amodel that distinguishes in which period the piece was composed. Several decision trees
have been generated to visualise the classification process. The importance of various
features is measured by the value of Mean Decrease Gini (MDG) that is calculated in the
random forests.

3

Results

As a result of the comparison of the OOB error rates (Table 1), the models using harmonic
and melodic features turned out to have better accuracy than the one using rhythmic
features. Table 2 shows the OOB error rates of the models that combine several feature
variables type. The discriminant models with harmonic and melodic features achieved thebest accuracy. Among two-period-models, the first and the third periods work the best as
the target categories.
Table 1: OOB error rate of each models
Feature variables typeunigrambigramtrigramfourgram

Harmony
MelodyRhythm0.34290.40000.68570.34290.34290.64290.42860.40000.7143–––

Table 2: OOB error rate of each models
Feature variables type

period 1 / 2 / 3Harmony (unigram + bigram) 0.3571
Harmony + Melody
0.2857
Harmony + Melody + Rhythm 0.3286

period 1 / 3

period 1 / 2

period 2 / 3

0.18000.08000.16000.27270.31820.27270.45650.39130.4130

We found out that the important harmonic features for the classification are the relative
frequencies of transition from the double dominant ninth chord with the root note omitted
to the dominant chord (#viio7 /V – V), the progression from the dominant seventh chord to
the tonic chord (V7 – I), and the second inversion of the dominant seventh chord (𝑉34 ). The

61

JADH2020

Short Paper - (4)important melodic features turned out to be the major second descent followed by theperfect fifth descent, the continuous chromatic ascent, and the repeat of same note after
the minor third ascent.

Fig. 1: Feature importance in three-period-model (left) and two-period-model (right)
Figure 2 is the visualisation of the classification process generated with a decision tree forthree periods. The relative error rate for this decision tree is 0.0227. The criteria for thediscrimination of the works composed in the first period are high frequency of transition
from the double dominant ninth chord with the root note omitted to the dominant chord,
the scarcity of major second descent followed by perfect fourth descent, the scarcity ofthe second inversion of submediant chord, and the abundance of chromatic ascent after
chromatic descent. The chord that discriminates the second period is the high frequencyof the first inversion of the tonic chord. Furthermore, the feature that sets the third period
apart is the transition from the first inversion of the tonic chord to the first inversion of thedominant seventh chord.

4

Conclusion and Future Works

In this study, we found out that the periodisation by Lenz (1855) is a valid classification
for the first period and the third period in terms of harmony and melody. However, themisclassifications frequently occur with respect to the second period. The accuracies of62

JADH2020

Short Paper - (4)Fig. 2: Decision tree for three periods with harmonic and melodic features
two-period-models also indicate that the model performs best when the second period is
not involved in the target category.
The question remains as to whether this result can be interpreted as the periodisation
criteria for other genres of his music. The precise boundary that separates the last period
from the rest of Beethoven’s early career still remains unclear. This is due to the lack ofworks composed during the 14 years (1810-1825) in which he did not compose any string
quartets. This indicates the necessity for further investigation that involves other genresof his music to compensate for this gap. Additionally, future research should compare theaccuracy of the models based on several periodisations other than Lenz (1855) in order toultimately redraw periodisation boundaries that best classifies the works by Beethoven interms of stylistic evolution.

References
1. Bücken, E. (1931). Ludwig van Beethoven. Athenaion.
2. Hirano, A. (2012). Ludwig van Beethoven (The Great Composers: Life and Works).
Ongaku-no-Tomosha.
3. Honingh, A. and Bod, R. (2011). “Clustering and classification of music by interval
categories.” International Conference on Mathematics and Computation in Music.
Springer.

63

JADH2020

Short Paper - (4)4. Kerman, J., Tyson, A., and Burnham, S. (2001). “Beethoven.” The New Grove
Dictionary of Music and Musicians. 2nd ed. Macmillan.
5. Lenz, W. v. (1855). Beethoven et Ses Trois Styles. Wentworth Press.
6. Moroi, S. (1983). Beethoven’s Piano Sonata: Study on Composition. Ongaku-noTomosha.
7. Neuwirth, M., Harasim, D., Moss, F.C., and Rohrmeier, M. (2018). “The AnnotatedBeethoven Corpus (ABC): A Dataset of Harmonic Analyses of All Beethoven String
Quartets.” Frontiers Digital Humanities, 5(16).
8. Richard, M. C. (2011). “Analyzing Tension and Drama in Beethoven’s First-Movement
Sonata Forms.” PhD diss. University of Toronto.
9. Schenk, E. (1967). “Ausgewählte Aufsätze, Reden und Vorträge.” Wiener musikwissenschaftliche Beiträge 7. Graz.

64

JADH2020

Short Paper - (5)Automated Face Detection for Pre-modern
Japanese Artworks using Deep Neural Networks
Alexis Mermet1,3 , Asanobu Kitamoto2,3 , Chikahiko Suzuki2,3 , and Akira
Takagishi4
1

Ecole Polytechnique Fédérale de Lausanne (EPFL)
2
ROIS-DS Center for Open Data in the Humanities
3
National Institute of Informatics (NII)
4

1

University of Tokyo

Introduction

In art history research, comparative style study, based on the visual comparison of characteristics in artworks, is a typical approach to answering research questions about works,
such as the identification of creators, the period of production, and the skill of the painter.
The creation of dataset useful for such tasks is a complicated and time consuming process
for art historians. The motivation of this paper is to help creating such collection by allowing efficient interaction between humans and computers. To achieve this goal, we areusing machine learning based solutions, more particularly, object detection, classificationand neural networks visualization methods. An useful starting point for such work is the
KaoKore dataset[1]. This collection consists of high quality, manually annotated, facial
expressions from pre-modern Japanese picture scrolls and picture books.

2

Methods

The creation of a dataset like KaoKore consists of two steps; namely the detection stepand the interpretation step. The detection step involves identifying the location of a face
object, while the interpretation step involves identifying the meaning of a face object,such as gender, position, status, and orientation of the face. To automate this process, weneed a face detector for the first step, and a face classifier for the second one. For bothof these tasks, our challenge is in the size of the fully annotated dataset, which is usually

65

JADH2020

Short Paper - (5)much smaller than expected for training a deep learning-based detector and classifier. This
problem is caused not only by the limited availability of open Japanese artwork images, butalso by the cost of manual annotation for a large number of artworks. Thus, our machinelearning algorithms should learn from small dataset and help human annotators to reducethe cost of annotation.
Our technical contribution for the first task is in employing a patching approach, that wecalled image patching, to solve the problem of small-scale and large-size image datasets.
Taking inspiration from research on style detection [2, 3] we implemented a detector using
patches of variable sizes that can be used as inputs for multiple deep learning architectures[4, 5, 6, 7]. The patching approach consists of the following (see figure 1): after padding
an image to a required size, we crop sub-images of a desired size from it, These sub-images,that we call patches, can either contain faces bounding boxes or not. Please refer to [8] forthe detail of the algorithm.

Fig. 1: Creation of patches of size 600*600 to input an SSD 300 backbone on an image fromthe Kaokore collection
The image patching method has the advantage of minimizing the loss of detail inducedby the resizing process that occurs before entering any deep learning architecture. Usingarchitectures with input layer of large enough dimensions, the patches created by the image
patching approach are of the same dimension as the input layer. Otherwise, when thedetector’s input dimensions are too small, the patches obtained with image patching canbe easily resized to the dimension expected by the detector while minimizing the loss ofdetail and preserving the patch’s aspect ratios. This approach also allows to extend thedataset by multiplying the number of artworks and faces that can be used while training
the detector.
For the second task of interpretation, we trained a classifier for the status of the face (seeSection Results). Moreover, to study graphical features of faces in pre-modern Japanese
artworks, we combined Guided Backpropagation algorithm[9] and the Grad-CAM algorithm[10] to keep track of the propagation of an object’s gradient through a network and66

JADH2020

Short Paper - (5)highlighted which pixels of the object triggered positive activation in the classifier.

3

Results

We first describe dataset split and evaluation metric that needs to be defined before experiments. The dataset split concerns how to divide the dataset into a training and testing
sets. In addition to a typical approach called random split, we employed new approaches
called inter-books split and intra-books split to take correlations within books/scrolls into
consideration [8]. In the intrabooks split, for example, each book/scroll should have its
images in both the training and testing sets so that machine learning algorithms can learn
from all books/scrolls in the dataset. On the other hand, evaluation metric is defined asfollows.
Recall is the number of properly detected faces among all the faces that should have been
detected.
Precision is the number of properly detected faces among all the faces that have been
detected.
Mean Average Precision, abridged to mAP, is a measure falling between 0 and 1 used tocompute the accuracy of an object detector at a given level. This level corresponds to thethreshold at which we consider that a predicted bounding box and a target bounding box
are overlapping or not.
For the detection step, we tried many combinations of deep learning models, and theresult indicates that a Faster R-CNN architecture[6], pretrained on ImageNet[11] and setup
on ResNet50[7], and patches of size 1333 * 800 is our best face detector on pre-modern
Japanese artworks, with a MaP score of 82.9% and a Recall of 91.1% on the Kaokore dataset.
The same deep learning architecture without patches only achieved a MaP score of 79.0%
and a Recall of 89.7% 89.7%. This suggests that the patching approach is effective fortraining a face detector.
For the interpretation step, we trained a ResNext[12] classifier to predict the status ofany face. This classifier attained an accuracy of 85.009% on all classes. It achieves highclassification score on the Nobles, Warriors and Monks class while having an hard time toclassify Commoners, showing that some status must have graphical representation rules
while other don’t. It also allowed to highlight relevant features for status classification,such as the ears, the hairstyles, the eyes and the hats.
Finally, to test the utility of our face detector in a realistic setup, we applied the detector
to a completely new images from Japanese painting scrolls, ‘Yugyo Shonin Engi-Emaki
Shojo-Kouji Kouhon’, archived in Shojo-Kouji Temple, Japan (hereafter we call it Kouhon
67

JADH2020

Short Paper - (5)dataset). Without any additional training, the detector obtained the list of bounding boxes
for faces from Kouhon dataset, as shown in Figure 2, and the results were then used by art
historians for complete annotation. Our approach reached a precision of 84.17% and a recallof 64.65% 64.65%. This result suggests that the total cost of the detection step is reduced
to only around 1/3 of its original cost in comparison to completely manual annotation.

4

Conclusion and future works

The most encouraging result is that the proposed face detector worked effective even forimages from new books (Kouhon dataset) without tuning parameters. This result suggeststhat our face detector can be used for expanding the horizon of research through the cycleof dataset expansion and model re-training for updating parameters. In the future, we
could improve our existing method for detection or even automatize the whole annotation
procedure including detection and interpretation. The interpretation task could be sped up
by a classifier so that art historians only have to care about hard cases that the network is
not able to interpret.

Fig. 2: Detection example on the Kouhon dataset using our best performing network (using
patches). Image courtesy of Shojo-Kouji Temple.

68

JADH2020

Short Paper - (5)Acknowledgment
Special appreciation goes to Shojo-Kouji and Yugyoji Museum for allowing us to use theKouhon dataset for collaborative research.

References[1] Y. Tian, C. Suzuki, T. Clanuwat, M. Bober-Irizar, A. Lamb, and A. Kitamoto, “Kaokore: A
pre-modern japanese art facial expression dataset,” 2020.
[2] C. Sandoval, E. Pirogova, and M. Lech, “Two-stage deep learning approach to the classification of fine-art paintings,” IEEE Access, vol. 7, pp. 41770–41781, 2019.
[3] R. Bai, H. Ling, Z. Kai, D. Qi, and Q. Wang, “Author recognition of fineart paintings,” in2019 Chinese Control Conference (CCC), pp. 8513–8518, 2019.
[4] Z. Cai and N. Vasconcelos, “Cascade r-cnn: Delving into high quality object detection,”
2017.
[5] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single
shot multibox detector,” Lecture Notes in Computer Science, p. 21–37, 2016.
[6] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection
with region proposal networks,” 2015.
[7] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image
recognition,” 2014.
[8] A. Mermet, A. Kitamoto, C. Suzuki, and A. Takagishi, “Face detection on pre-modernjapanese artworks using r-cnn and image patching for semiautomatic annotation,” in SUMAC
2020 - 2nd Workshop on Structuring and Understanding of Multimedia heritAge Contents,
in press.
[9] J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller, “Striving for simplicity: The
all convolutional net,” 2014.
[10] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam:
Visual explanations from deep networks via gradientbased localization,” International
Journal of Computer Vision, vol. 128, p. 336–359, Oct 2019.
[11] J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition,
pp. 248–255, 2009.

69

JADH2020

Short Paper - (5)[12] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations fordeep neural networks,” 2016.

70

JADH2020

Poster - (1)

GIS Visualisation for the Administrative Document
Catalogue of the Government-General of Taiwan
Hajime Murai1 and Toshio Kawashima1
1

1

Future University Hakodate

Introduction

The digitisation of official government documents is important from the viewpoint ofhistory, archives, and digital humanities. However, only little digitisation and publicationof historical government documents has been done in Japan. There are several reasonsfor this (Koga, 2005). First, Japan’s frequent natural disasters make it difficult to preserve
documents. Moreover, various documents were lost during the Second World War due to
air raids, evacuations, and disposal (National Archives of Japan, 2006).
Two of the few exceptions are government documents from the governors general of Taiwan
and Korea, which have been preserved in exceptionally good conditions (Kato, 2002). The
former includes administrative documents depicting public organisations in pre-war Japan.
Unlike most official Japanese documents, this collection has been published as image dataand partially digitised in text format (Higashiyama, 2017).
The goal of this study is to develop a tool for visually grasping an overview of the enormous
amounts of administrative documents of Taiwan. Moreover, it is also the aim of this study toinvestigate the usefulness of digitally archived text data of official documents for historicaland social studies. For this purpose, an automatic KML data construction application for
GIS visualisation systems has been developed.

2

Target contents

This study examines data from the ‘Catalog Database of the Government-General of Taiwan’s
Administrative Documents’ (ISSCU, 2015), a database outlining the documents found in the‘Catalog of the Government-General of Taiwan’s Administrative Documents’ (台湾総督府文書目録, CCCADGT, 1994). These text data include the titles of official documents from 1895

71

JADH2020

Poster - (1)to 1914.
In addition to the catalogue documents, three types of datasets were created for automatic
KML data construction. The first is a dictionary for old and historical words used in the target
documents. The second is a word category for historical topics (Murai and Kawashima,2018), and the third is a dataset for old place names and positions in Taiwan.
The dictionary for old and historical words in the Catalog Database of the GovernmentGeneral of Taiwan’s Administrative Documents includes 136 words related to places and61 historical or cultural words. These words were added to a normal Japanese dictionary
for morphological analysis, and the texts were divided into words. The word category forhistorical topics includes 785 words from 26 topics (see Table 1). This category was utilisedfor detecting topics within each official document title (Murai and Kawashima, 2018).
The dataset for old place names in Taiwan includes 274 pairs (see Table 2) of place names
and geographic information (latitude and longitude).
Table 1. Number of words in topic categories
Topic category

Number of words

Topic category

Number of words

Personnel shift
Publicity
Rules
Office work
Permission
HygieneLivestock
Economy and tax
Judgment
Infrastructure
PoliceCorrespondence
Education

119
29
29
20
8
91
18
36
53
21
16
14
26

Monopoly
Establishment and closure
Traffic
Disaster and welfare
Mining and engineering
Rebellion and indigenous
Military
Agriculture
International
Residents
Imperial family and rituals
ReligionFisheries

16
13
35
22
12
22
56
33
25
7
28
29
7

Table 2. Number of place name and global positions
Area

Number of place names

Area

Number of place names

台北 (Taipei)

32
26
29

南投 (Nantou)

11
15
18

台中 (Taichung)
台南 (Tainan)

雲林 (Yunlin)
嘉義 (Chiayi)

72

JADH2020
Area

Poster - (1)Number of place names

桃園 (Taoyuan)

7
高雄 (Kaohsiung) 30
基隆 (Keelung)3
新竹 (Hsinchu)8
苗栗 (Miaoli)13
彰化 (Changhua) 25

Area

Number of place names

屏東 (Pingtung)

16
9
10
15
7

宜蘭 (Yilan)

花蓮 (Hualien)
台東 (Taitung)澎湖 (Wuhu)

These names were extracted manually from the ‘Catalog of the Government-General of
Taiwan’s Administrative Documents’.

3

KML construction

The method for extracting topics and geographic information based on the three typesof databases is depicted in Figure 1. Moreover, the target titles of official documents also
contain chronological information since official documents include promulgation dates.
Therefore, the extracted topic categories and geographical information with chronological information were combined into the KML data format. As a result, users can visually
recognise chronological and spatial shifts in historical topics by utilising the GIS application(Figure 2).

4

Conclusions and future work

The datasets including the topic word categories and place names enable the extraction ofchronological and spatial characteristics for historical documents. Moreover, KML data enable users to visually recognise the geographical characteristics as well as the chronologicaland spatial shifts in historical topics.
Although there are various limitations to a catalogue-only analysis, the Catalog of theGovernment-General of Taiwan demonstrated the utility of this dataset.
One of possible future works may be utilization of named entity recognition based onnatural language processing techniques. However, the accuracy is not enough at this stage,
therefore, it is necessary to wait for future development of these field.
If appropriate datasets of old place names, geographical information, and words thatindicate historical topics are constructed, this method and software would be applicablefor other historical documents. Also there are other valuable data formats for GIS, it may be
useful to support multiple formats for generalization.
73

JADH2020

Poster - (1)Fig. 1: Method for extracting topics and geographic information

References
Catalog Compilation Committee for the Administrative Document of GovernmentGeneral of Taiwan (CCCADGT), (1994). Catalog of the Government General of Taiwan’s
Administrative Documents (In Japanese: 台湾総督府文書目録). Chukyo University: YumaniShobo, 1994.
Higashiyama, K., (2017). “Management of Administrative Document in the Periods of
Taiwan Sotokufu” (In Japanese: 草創期における台湾総督府の文書管理). Bulletin of Social
Science Research, Institute of Social Sciences of Chukyo University, 38(1): 51–105.
Institute of Social Sciences of Chukyo University (ISSCU), (2015). “Catalog Databasefor the Government General of Taiwan’s Administrative Documents” (In Japanese, 台湾総督府文書目録データベース). CD-ROM.
Kato, K., (2002). “Defeat and Destruction of Official Documents: The Actual Situation in
the Colony and Occupation Period” (In Japanese: 敗戦と公文書廃棄-植民地・占領期における実態). The Bulletin of the Department of Historical Documents, 33: 105–144.
Koga, T., (2005). “Government Information and Roles of Libraries and Archives: Recent
Policy Issues in Japan”, Progress in Informatics, 1: 47–58.

74

JADH2020

Poster - (1)Fig. 2: Example of extracted KML on Google Earth75

JADH2020

Poster - (1)Murai, H., Kawashima, T., (2018). “Towards for Quantitative Analysis of Official Document
Catalog of the Government-General of Formosa”, Journal of Japan Society of Information
and Knowledge, 28(2): 79–84.
National Archives of Japan, (2006). “Preserving and Using Important Records: A Trial of
the Digital Archive in National Archives of Japan” (In Japanese, 重要記録の保存と利用: 国立公文書館デジタルアーカイブズの試み). Archives, 22: 26–29.

76

JADH2020

Poster - (2)Educational utilization of digital archive-Inquiry based curation learning in on-line educationMasao Oi1
1

1

University of Tokyo

Background

I. Irich criticized passive learning by linking it with social structure, and proposed to regain
proactive learning 1 .
1. It has been pointed out that proactive learning is lost due to the lack of “selfrelevance”2 .
Proactive learning is one the main axes in the Japanese new curriculum guideline which
was fully implemented from April 2020. In order to realize such learning, importance wasplaced on
2. developing skills of students to collect information by themselves using Information
and Communication Technology (ICT) and3. utilize facilities such as museum, library and archives (MLA) to develop multifaceted
and multi-angular perspective.

2

Problems
1. Teaching materials based on local history and oral history is an example putting
importance on “self-relevance” learning. However, such subjects are still far from thedaily lives of students, causing temporal and spatial distances. (Thus it can not fully
function as a material to induce “self-relevance”.)
2. In regards to the training of information utilization ability of ICT, the educational goalis ”the comprehensive learning of information collection, analysis, and transmission”.
However, there are criticisms that many practices only covers the instruction of ICT
equipment utilization 3 . There are ICT utilization practices that focus on nurturing

77

JADH2020

Poster - (2)thinking ability, yet, since information is “given” by teachers, learning is rather passive
for children and students.
Thus, opportunity to think based on the multifaceted and multi-angular information
is not given.
3. In regards to the use of MLA, it is suggested that it is important to visit related facilities
4(to depthen the understanding of such places). However, lack in time and distance isa burden for many teachers. As a method to solve such problems, there are practicalstudies using digital archives such as application of the Asian Historical Materials
Center and the Niigata regional video archive5-6 . Yet, such studies are limited toindividual and only single archives, therefore the materials and information are
restricted to limited resources. Under such circumstances, students are unable toencounter various materials and is difficult to notice the multifaceted nature of events.

3

Solution

Therefore, in order to solve the above issues, this research has developed teaching materials,
designed lessons, and practiced from the following viewpoints.
1. In order to solve the issue of ”lack in self-relevance”, the temporal and spatial distances
between the learning target, and the learner should be reduced.
Reduction of such distance will lead to familiarity and connectivity, which gives
opportunity to think more deeply towards the learning object.
2. Passive learning due to the given teaching materials can be solved through the “curation” activity which is collecting, organizing information and materials, and adding
new meanings and values towards the learning object.
3. In order to create more occasions for students to encounter serendipity (unexpectedencounters and surprises) using multiple archives with diverse materials and opinions
which leads to thinking from multifaceted perspectives is important.

4

Research method

A curation class was designed which used digital archives to consider history in a multifaceted and multi angled manner, multiple sources was used and practiced an inquiry-basedlearning classes at elementary, middle and high schools.
1. Practiced a workshop which utilized the ”Japan Search” which is integrated material
search digital archive as preliminary practice.
78

JADH2020

Poster - (2)2. Implementation of a new function was made on the “Japan Search” along with theconsultation of development engineers to improve functions for educational usage.
By implementing such function, collaborative collection, collaborative editing, collaborative browsing and online browsing of materials became possible.
3. Constructed question from textbook materials and developed the teaching materials and class design along with the usage of ”Japan Search” in accordance to theindividual theme.
4. Practiced social studies classes throughout the year at elementary, middle and highschool.
5. Rubric’s analysis on cognitive transformation of children was conducted before andafter the practice.

5

Result & Future tasks

By setting the theme based on one’s own interest and creating questions as a starting point
to collect materials using the integrated archive, made recognitions that the joy of exploring
based on the interests and the viewpoints of reading and interpreting the materials varied
depending on the person.
An integrated archives platform that is easy for anyone to use in classes, with accumulated
practical examples of each unit that are closely connected to the annual curriculum andprovided with educational metadata such as materials and connections with each unit ishoped to be constructed and completed.

6

Current research and future prospect

Future prospect of this research is to develop an online teaching platform to support the richlearning of distance education in order to create fair and diverse educational opportunities.
This research will contribute to inquiring support for the learning of students and thedevelopment of teaching materials for teachers.
In order to realize such concept, specifying on which metadata to use and how to promote
cooperation with related organization is the future issue.
The presentation will be reporting the result of the lesson practiced using digital archive thatwas conducted throughout the year. Remote digital archive was used during the COVID-19
lock down from April 2020 to continue this study.

79

JADH2020

Poster - (2)Therefore, the presentation will not only focus on how to utilize such digital archives foreducational purposes, but also how it can ensure fair and equitable access to educational
opportunities under severe conditions. For instance, educational method that can alsofunction under fatal pandemic will be required in up coming era. Thus, the social significance of this research will also be discussed, report practical research with high novelty
that bridges various future education and digital technology.

Fig. 1: Curation learning in face-to-face class

Fig. 2: Curation learning in online class

7

Bibliographic references

1 Illich, Ivan. (1977). “The deschooling society.” (Azuma, Hiroshi., Ozawa, Syuuzou.). Tokyo
Sougensya.
2 Suwa, Masaki., Fujii, Haruyuki. (2015). “Chi no dezain Zibungototoshite kangaeyou.”
Kindaikagakusya.
3 Ueda, Yuta. (2013). “Consideration About the Need for Use of Archives in Education of theInformation Literacy Education in High School.” Records Management, No65, pp.100-108.
4 Nagai, Hiroshi. (2004). “The Significance of Archives of Social Studies Education.” Shakaika
Kyouiku Kenkyu, No.91, pp.34-40.

80

JADH2020

Poster - (2)5 Yoshimine, Shigeki. (2007). “Asia Rekeshi Shiryou Center to Koukou Rekishi Kyouiku.”
Archives, 27: 72-75.
6 Kitamura, Yorio. (2018). “Practical Research of Using Local Digital Visual Archives in
Education: Possibilities and Problems.” Digital Archive Gakkaishi, Vol.2, No.2, pp.83-86.

81

JADH2020

Poster - (3)“Noh as Intermedia”/Intermedia Analyses of Noh
Theater Plays
Takanori Fujita1 , Jarosław Kapuściński2 , François Rose2 , Scott Bailey3 , Peter
Broadwell4 , Simon Wiles4 , and Glen Worthey5
1

Kyoto City University of Arts
2
Stanford University
3
North Carolina State University Libraries
4

Stanford University Libraries
5
HathiTrust Research Center

The Noh as Intermedia project (noh.stanford.edu) uses web-based technologies for interactive annotation and visualization of performing arts materials to investigate and present
how Noh theater exemplifies intermedia. “Intermedia” here is defined as a way of forming
expression that draws on relationships between art forms (literature, vocal and instrumental music, stage movement, dance, masks, costumes, and props, in the case of Noh) andtheir cumulative impact.
To untangle some of the unique “coming together” of artistic media in Noh, the projectsite builds a multi-scale, “thick” analysis around high-resolution video recordings of live
performances of two exemplary and contrasting Noh plays, both likely from the late 14th
century CE: the feminine-centered “wig” play Hashitomi, with its evocations of poetry andscenes from Genji monogatari, and the miraculous “god” play Kokaji, which illustrates its
tale of the forging of an enchanted sword with intricate layerings of dance, music andcostumes. The site’s examination of the two plays and the Noh genre itself moves from highlevel essays describing the history and aesthetics of Noh to a middle-level set of interactive
catalogs, descriptions, and image, video and audio examples presenting the varied andoften highly systematized media elements that comprise a Noh performance. These include
theatrical and dramatic forms at the macro and meso scales, poetic modes of declamationand singing, dance patterns and other gestures, instruments, rhythmic and melodic modes
and musical forms, as well as costumes, masks, stage positioning and props.

82

JADH2020

Poster - (3)Fig. 1: A faceted catalog of shōdan (formal sections of a play)

83

JADH2020

Poster - (3)At the most finely detailed level, the project provides a novel playback and annotation
environment that is synchronized at subsecond-level granularity with the streaming video
playback, combining plot synopses, visualizations of formal sections, and synchronizedlibretto translations with a multilayered, multimedia score that encodes every sung or
spoken word, musical element, and stage action. This score thus represents, for a specificperformance, even the components of a play that are not traditionally notated, such as
elements of the musical accompaniment. In an analogous manner, this entire mode ofpresentation has the effect of exposing, in an immediately engaging and comprehensible
way, the myriad subtle details of a Noh performance whose interplay across media types
lends the performances their extraordinary richness, yet which would otherwise likely go
unremarked by those who are not already experts in the genre. The site’s use of English as its
primary language, providing transliterated romaji texts and translations whenever possible,
furthers this pedagogic aim, complementing the rich lineage of Japanese-language Nohscholarship and contributing to the steadily growing body of analyses in other languages.

Fig. 2: The interactive playback environment, with synchronized score, text, formal map
and description
The site also engages with the tensions between providing interactive, multimedia visualization and annotation features that are highly tailored to a specific performance genre (Noh,in this case) versus having the flexibility to accommodate other genres and art forms within
the same framework. In this project, the investigators found that favoring the specific overthe general provided the greatest degree of interpretive insight, but doing so consequently
requires a greater effort to integrate other genres, particularly if these other forms are to be84

JADH2020

Poster - (3)presented at equivalent levels of coherence and clarity.
A primary analytical feature of the site and project is the explication of a ranking system
for the shōdan (formal sections of a play) that assigns higher values to sections exhibiting
greater intermediality. A section with poetic text, congruent music and dance receives
a higher “index of intermedia” (IoI) ranking than a section with fewer such modes, andwithin each modality are further gradations, e.g., a section with rhythmically congruent
chanting receives a higher IoI value than one that features unmetered speaking, due tothe former’s increased energy and use in moments of dramatic climax. While turning richtheatrical experience into numbers and graphs risks being reductive, it can be an informative
abstraction for comparing the profile and flow of intermedia development between plays.
The interactive visualizations on the site present these evaluations of intermediality by wayof graphical shōdan “maps” or timelines of the performances in which a section’s widthcorresponds to its relative duration and its height represents its intermediality ranking.
Observing these visualizations in the context of the richly layered explicatory apparatus
surrounding the live recordings facilitates a greater appreciation, for example, of how themacro-scale dramatic trajectory of Hashitomi utilizes a gradual accretion of interlinked
media elements to build to a dramatic apotheosis, while Kokaji exhibits a greater ebb andflow between sections of high and low intermediality across its two acts. The interface
explicitly seeks to hold together, and in dialectic, multiple presentations of a single originary
phenomenon: the video, the timelines, the scores, etc., all as manifestations of the play,according to different modes/intentionalities. In this way, the interface itself could be seen
as a form of intermedia presentation of the deeply intermedic art form of Noh.

1

References

Bethe, Monica, and Karen Brazell. Dance in the Nō Theater. Ithaca, N.Y.: China-Japan
Program, Cornell University, 1982.
Chion, Michel. L’audio-Vision: Son et Image Au Cinéma. Nathan, 1990. Later translated as
Chion, Michel, and Claudia Gorbman. Audio-Vision: Sound on Screen. Columbia University
Press, 1994.
Cook, Nicholas. Analysing Musical Multimedia. Clarendon Press, 1998.
Hare, Thomas B. Zeami’s Style: The Noh Plays of Zeami Motokiyo. Stanford, Calif: Stanford
University Press, 1986.
Higgins, Dick with an appendix by Hannah Higgins. “Synesthesia and Intersenses: Intermedia.” Something Else Newsletter 1. Something Else Press, 1966.
Japanese Performing Arts Research Consortium. “Nōgaku 能楽.” https://jparc.online/nog
85

JADH2020

Poster - (3)aku/, 2020.
Konparu, Kunio. The Noh Theater: Principles and Perspectives. Warren, Conn: Floating
World, 2005.
Tamba, Akira. The Musical Structure of Nō. Tokyo: Tokai University Press, 1981. Originally
Tamba, Akira. La Structure Musicale Du Nô: Théâtre Traditionnel Japonais. Paris: Klincksieck,
1974.

86

JADH2020

Poster - (4)A fundamental study of effective visualization anddescription of longitudinal changes of classicalJapanese poetic vocabulary
Hilofumi Yamamoto1 and Bor Hodošček2
1

Tokyo Institute of Technology
2

Osaka University

The purpose of the current project is to report on the development of a system that visualizes
the co-occurrence patterns of words in the Kokinshu (ca. 905). There are not many studies
exploiting visualization technology for ancient languages, with researchers in many cases
continuing to carefully analyze limited material through visual inspection. Within thiscontext, the purpose of this study is the longitudinal analysis of ancient languages, and thedevelopment of an effective visualization system tailored to them. To that purpose, we will
first prepare the vocabulary database of the ancient language, and then analyze the ancientlanguage through a visualization providing a bird’s-eye view of present classical vocabulary.
Unlike modern languages, there is not a massive amount of ancient language data, and wemust perform research within that limitation. In addition to that, considering that existing
text expresses limited meaning and content, there are many differences between modern
and ancient languages in the analyses. Uniformly solving the problem is not straightforward.
In this study, we limit the database to the Kokinshu, which is the first emperor-orderedanthology of Japanese classical poetry (ca. 905) consisting of 1,111 poems. We will use
the poem texts and their contemporary language translations to examine whether the
analytical process is proper and valid. Using poetry data, we will 1) calculate the similarityof each word, and 2) generate word relationship data between any two words (Mikolov etal. 2013). We have studied the extraction of relational pairs of ‘orange,’ ‘plum,’ and ‘cherry’flowers in poetic Japanese (Hodošček and Yamamoto and, 2017). In this study, we staticallygenerated 2-gram patterns of poem words without calculating each word’s similarity inadvance. In the present study, however, we perform an exhaustive investigation of related
pairs rather than investigate particular words one at a time. We will examine the differences
between calculating and not calculating the similarity of each word, and consider whether

87

JADH2020

Poster - (4)it is possible to display dynamic word relationships without the thesaurus showing the
vocabulary category explicitly.
We will apply the following procedure: first, the machine-learned processor evaluates thelength of the appropriateness of the ancient language word; secondly, we will evaluate
the classification of the meaning of poetic words; finally, according to the classification
categories of poetic words, we will perform generation of co-occurrence patterns, andattempt to visualize. For determining the word units, we will set up the following four casesof co-occurrence patterns and examine if it is possible to generate the patterns according to
the cases. I.e., 1) word pairs are adjacent, and a. the order is probabilistically constrained,b. the order is not probabilistically constrained; 2) word pairs are not adjacent, and a. theorder is probabilistically constrained, b. the order is not probabilistically constrained.
An unsolved problem using machine learning for natural language processing tasks is thatit cannot satisfactorily distinguish lexical similarity from lexical relevance. (Hill et al. 2015)
As an example of the oftentimes disjoint relationship between lexical similarity and lexical
relevance: ‘plum’ and ‘cherry’ are similar to each other, but there is no relevance between
them other than that they are flowers. On the other hand, ‘plum’ and ‘branch’ are both
associated as parts of plants, but are otherwise lexically dissimilar. If machine processing
can separately recognize similarity and relevance, the modeling of vocabulary would becloser to a human experts, and that will contribute to the processing of classical vocabulary
as well. The analysis and visualisation system will use the Kokinshu database of Nakamura
et al. (1999) as the corpus.
The system of the present study is a working in progress, and the present paper will carryout the interim report. This work was supported by KAKENHI (18K00528), Grant-in-Aid for
Scientific Research (C).

References
1. Hill, F., Reichart, R., and Korhonen, A. (2015). SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation. Computational Linguistics, 41(4),665–695. https://doi.org/10.1162/COLI_a_00237
2. Hodošček, Bor, and Hilofumi Yamamoto (2017) A study on the extraction of relational pairs of ‘orange,’ ‘plum,’ and ‘cherry’ flowers in poetic Japanese. Symposiumof Humanities and Computer, Proceedings of Humanities and Computer, Vol. 2017,
No. 2, pp. 207–212, 12.
3. Mikolov, Tomas, Kai Chen, Greg Corrado, and Jeffrey Dean (2013) ”Efficient Estimation of Word Representations in Vector Space” CoRR, URL: http://arxiv.org/abs/13
01.378188

JADH2020

Poster - (4)4. Nakamura, Yasuo, Yoshihiko Tachikawa, and Mayuko Sugita (1999) Kokubungaku kenkyu shiryokan detabesu koten korekushon ”Nijuichidaishu” Shohobanbon CD-ROM (Database Collection by National Institute of Japanese Literature ”Nijuichidaishu” the Shoho edition CD-ROM; Iwanami Shoten.

89

JADH2020

Poster - (5)Analysis of difference between male and female
facial expressions in Japanese picture scrolls using
GM Method with IIIF Curation Platform
Chikahiko Suzuki1,2 , Akira Takagishi3 , Alexis Mermet4,2 , Asanobu
Kitamoto1,2 , and Jun Homma5
1

ROIS-DS Center for Open Data in the Humanities
2
National Institute of Informatics
3
University of Tokyo
4
L’Ecole polytechnique fédérale de Lausanne
5

1

FLX Style

Introduction

The purpose of the paper is to analyze the difference of facial expressions between maleand female in picture scrolls created by multiple painters in the middle ages of Japan (13thto 16th centuries), and to clarify the production situation.

Fig. 1: Gender with different codes (left) and same codes (right).
In the Japanese picture scrolls of that time, male and female are drawn differently by
combining codes. For example, warriors (武士) and nobles (貴族) are distinguished between

90

JADH2020

Poster - (5)gender by codes such as clothes and hairstyles. However, the monks (僧侶) and nuns (尼僧)
are ambiguous because they have basically same codes (Fig1). Hence monks and nuns canonly be distinguished by facial expressions.
Analysis of facial expressions is also important to restore the original works. Because many
manuscripts were copied in the Middle Ages of Japan. Research questions, such as theindividuality of the characters in the original works, can be answered by examining facial
expressions in the copied manuscripts.
To analyze the multiple facial expressions, the following way has been traditionally used.
First, place a labeled photo that capture all or part of art works on a large table. Second,
rearrange photos for comparison and grouping. We call this method “GM method”. Weshow that digital technology, such as IIIF Curation Platform (ICP) (Kitamoto et al., 2018)
and the machine learning with KaoKore Dataset (Tian et al., 2020), greatly enhances theusefulness and potential of the GM method.
The name GM method comes from the Italian art historian Giovanni Morelli (1816-1891) who
is the founder of the style comparative study of art history with special attention to detailsof art works. GM also stands for Gazing Microcontents to take advantage of fragmented
content (micro-content) for research (Suzuki and Kitamoto, 2019).

2

Materials

The target of this paper is one copy of Yugyo Shounin Engi-Emaki, “Shojo-Kouji Kouhon”
遊行上人縁起絵巻 清浄光寺 甲本 (Kouhon) archived in Shojo-Kouji Temple. The originaland lost version of Yugyo Shounin Engi-Emaki is a 10-volume picture scroll depicting theestablishment of the Jishu 時宗 sect of Buddhism in the Kamakura period (12th to 14th).
Yugyo Shounin Engi-Emaki was copied many times during the early 14th and 17th centuries.
According to previous research, multiple painters shared the work of Kouhon and modern
and classical styles are mixed (Takagishi, 2020).
We have a particular interest in Yugyo Shounin Engi-Emaki in the context of the Jishu sect.
In the Middle Ages, Jishu has been criticized for mixing different genders, namely monks
and nuns, in the same group. Therefore, in the every copy of Yugyo Shounin Engi-Emaki,
monks and nuns are drawn on the same scene, but are arranged in two separate groups
with clear signs such as Juuni-Kou-Bako 十二光筥 to claim that Jishu is a disciplined sect(Fig2). These signs allow us to distinguish monks and nuns having the same code.
Kouhon has 28 scenes where separated monks and nuns are drawn. According to previous
research, these scenes were drawn by painter A and B (both names are unknown) (Iwase,1989). By focusing on difference between painter A and B in terms of facial expressions of91

JADH2020

Poster - (5)Fig. 2: Two groups with Juuni-Kou-Bako (three-colored lid box in the center).
monks and nuns, these scenes allow us to analyze how Kouhon was copied from the lost
original work.

3

Results

The GM method with ICP consists of two steps. Frist, we used ICP to create curations for
all facial expressions in Kouhon. At this step, we have two tasks, 1) to draw rectangles
surrounding each facial expression, 2) to add basic metadata (gender, status, direction,
source) of facial expressions. These two tasks should be performed by human experts, but
the first task could be done faster using a machine learning-based face detector. The effectof machine learning for reduction of annotation time is discussed in another paper (Mermet
et al., 2020).
Second, we analyzed 579 facial expressions appeared in the 28 scenes. At this step, we used
the IIIF Curation Board (ICBoard). ICBoard shows an element of a curation like a card placedon a flat mat, which can be moved or arranged. ICBoard has many functions to help the GM
method such as grouping mats and color-coded stickers.
By rearranging 579 facial expression, difference in drawing style of two painters became
clear. Painter A draws a unique facial expression in deep colors. Both genders are drawn
similar. The outlines of both monks and nuns have smooth curves. The variation withinsame gender is small. Painter B draws in light colors, the differences between genders areapparent. The outlines of the monks are rugged, while the nuns are smooth. The variation
within gender is large, such as the hair shaving marks (Fig3).

92

JADH2020

Poster - (5)Fig. 3: Facial Expression by painter A (above) and painter B (below). Marked: Red = Painter
A, Blue = Painter B, Yellow = Monk

93

JADH2020

Poster - (5)This result suggests different attitude of two painters on copying the lost original scroll.
Although Painter A copied of the original composition, added his unique style. On the other
hand, painter B draw different facial expressions between monks and nuns because hetried to faithfully copy the original works. We argue that monks and nuns might have been
individually identified in the original picture scroll as illustrated by painter B.

4Discussion

Using the GM method with digital technology, a large number of facial expressions weresmoothly analyzed. Analysis of this scale, such as 500 photos, has been hard for real photos
but the digital method using ICP allows us to embark on a comprehensive analysis of facial
expressions. In addition, the GM method can be more powerful using machine learning
such as object detection and classification algorithms.
Our results support the previous study that Kouhon was created by multiple painters. Furthermore, this result brings about a new research question as follows. Similar facial expressions appear in different scenes by painter B. If painter B faithfully copied the charactersin the original, these similar characters have been intentionally drawn in the lost original
picture scroll as the important person. This is an important both for art history and Jishu
sect.

5

Acknowledgments

We thank the Shojo-Kouji Temple and Yugyoji Museum for allowing us to use images for thisresearch.

References
Kitamoto, A., Homma, J. and Saier, T. (2018) “IIIF Curation Platform: Next Generation IIIF
Open Platform Supporting User-Driven Image Sharing.” Jinmoncom Symposyum 2018 :327-334.
Tian, Y., Suzuki, C., Clanuwat, T., Bober-Irizar, M., Lamb, A., and Kitamoto, A. (2020).
“KaoKore: A Pre-modern Japanese Art Facial Expression Dataset.” arXiv: 2002.08595.
Suzuki, C., and Kitamoto, A. (2019). “Using Pre-Modern Japanese Book 江戸買物独案内 as
Microcontents for Reconstruction the City of Edo.” Jinmoncom Symposyum 2019 : 11-18.
Takagishi, A. (2020). Reflections on medieval Yamato-e. Tokyo: Yoshikawa-koubunkan.

94

JADH2020

Poster - (5)Iwase, H. (1989). “Yugyo-Shonin-Engi-Emaki; Seijoko-Ji-Bon ni tsuite.” Ars buddhica (185):
p51-59Mermet, A., Kitamoto, A., Suzuki, C. and Takagishi, A. (2020). “Automated Face Detection forPre-modern Japanese Artworks using Deep Neural Networks.” JADH2020 (in press)
IIIF Curation Board http://codh.rois.ac.jp/software/iiif-curation-board/95

J�������
A���������� ��
D������
H���������

JADH2020ワークショップ

人文系大学院における情報リテラシーの在り方
Workshop on The State of Digital Literacy inJapanese Humanities Graduate Programs
2020年11⽉20日
オンライン開催