IIIF Curation Platform
as a Research Toolof Microcontents in the Humanities
14th CODH SeminarFebruary 18, 2021
Chikahiko SUZUKI
ROIS-DS Center for Open Data in the Humanities
National Institute of Informatics

Personal introduction
Chikahiko SUZUKI
Assistant Professor by Special Appointment, ROIS-DS Center for Open Data in theHumanities
Also working at the National Institute of Informatics

Fields of expertise: Digital Humanities, Cultural Resources Studies
I use The IIIF Curation Platform mainly for converting humanities materials into
microcontent (described later).

Recipe content: Focusing on examples of useby humanities researchersWhile the director Kitamoto gave an overview of the ICP features, I will present
mainly ICP use cases.
1. The microcontent named “Collection of Facial Expressions“, and art history
research using ICP
2. Group work and curathon using ICP
3. The use environment and environment setting of ICPCollectionof FacialExpressions(KaoKore)Collection of Facial Expression (KaoKore)
http://codh.rois.ac.jp/face/•A collection of facial expressions, appearing inartworks, distributed via IIIF by various institutions.
•Facial expressions are organized and can besearched•Facial expressions are collected primarily fromJapanese picture scrolls and illustrated books•Metadata is provided for all facial expressions•8,845 facial expressions from 299 works,(As of February 2021)

KaoKore and the IIIF Curation Platform
Almost all of the functions of the IIIF Curation Platform are used in KaoKore.
● Cropping of faces with the IIIF Curation Viewer● Addition of metadata using the IIIF Curation Viewer*Batch addition of metadata labes using the IIIF Curation Editor● Management using the IIIF Curation Manager and editing using the IIIFCuration Editor● Facet search using the IIIF Curation Finder● Linkage with machine learning, and dataset construction● Organization, analysis, and research using the IIIF Curation Board

IIIF Curation Viewer

1. Image display ★
Display in the IIIF
Curation Viewerof artwork from whichfaces are collected
A IIIF manifest isdragged & dropped fromthe website or archive ofeach facility. ICViewer
enables us to load
images simply.

IIIF Curation Viewer

2. Image collection ★
Using ■ ”Crop” button and☆ “favorite” button tocollect facial expression ina curation list

IIIF Curation Viewer

3. Saving a curation★

Using “Export” button to saves acuration of facial expressions.
*At this stage, curation is saved in
JsonKeeper. We can share curation
using a URL.

IIIF Curation Editor

4. Management by naming★

We can rename each curation.
With own name, we can eacly
manage curations
Using the IIIF Curation Editor, we
can rename curations’ nameisplayed on the IIIF Curation
Manager. The character string in“label”, directly under “object”
section, serves as the title.
*The initial setting name is"Curating list"

IIIF Curation Viewer

5. Adding metadata★

Metadata is added with "Editcuration metadata" button
Metadata of the KaoKore: face
direction, gender, social class...
… However, creating items oneby-one for each image using "+item" button is burden for theresearchers, so..

IIIF Curation Editor

6. Batch storage of metadata items
We can use the IIIF
Curation Editor to Batch
setting of metadata items.
*Batch replacement canalso be done with Ctrl+F inthe ICEditor.
In the example at right, acomplex replacement isperformed using an
external editor (Oxygen)★＋0.5

IIIF Curation Viewer

7. Suspension of work ★
"Overwrite update“ button (exportwith overwrite) is performed in orderto temporarily suspend the curation(both face cropping or metadatastage)*Caution: When you added metadata,
it will not be saved correctly if you
didn’t export from the metadata input
fieldIIIF Curation Manager

8. Management of group work ★
Management of group work with ICP
The KaoKore is constructed bymultiple workers.
To support group work, theadministrator ascertains the login
account of each worker, and manages
curation from the ICManager

User A

User BIIIF Curation Finder / Canvas Indexer

9. Search with metadata★

The IIIF Curation Finder enables us tosearch curation using metadata
Search results can be switched between
curation or canvas units
http://codh.rois.ac.jp/face/iiif-curation-finder/
The example results of a canvas searchusing value "Buddhist monk" are shown atrightIIIF Curation Finder

10. Derived curations★

A derived curation can be createdby selecting the "+" button of IIIF
Curation Finder. Selected images
can be exported with own URL.
In the KaoKore, this function help
us to curation list of faces ofinterest and similar faces.
*This is also a classical method forstyles comparative studies in arthistory

IIIF Curation Finder / Canvas Indexer

11. Batch acquisition of information (1)★★

Batch checking of information
displayed in the IIIF Curation Finder(metadata, image coordinates)
We can use searchEndpointUrl andfacetsEndpointUrl*This is also used for KaoKore dataset
construction described latersearchEndpointUrl
Endpoint URL for Canvas
Indexer search→Enables checking ofindividual images/metadata ineach contained curation

Content of curation

IIIF Curation Finder / Canvas Indexer

12 . Batch acquisition of information (2)★★

Batch checking of information
displayed in the IIIF Curation Finder(metadata, image coordinates)
We can use searchEndpointUrl andfacetsEndpointUrl*This is also used for KaoKore dataset
construction described later

facetsEndpointUrl
Endpoint URL for acquisition ofCanvas Indexer facet list
→Enables acquisition of
metadata list and number of
each metadata valueOther

13. Use in machine learning ★★★
"KaoKore Dataset"
By linking with the IIIF Curation Platform,
all facial expression images and metadatafrom the KaoKore are reconstructed asdataset for machine learning called the"KaoKore Dataset"
http://codh.rois.ac.jp/face/dataset/

Facial expressions in the style of Japanese picture
scrolls, automatically generated through machinelearning

IIIF Curation Viewer

14. Rectangle selection throughcoordinate input ★
Use of the KaoKore Dataset● Automatic recognition of facial expressions from picture●scrolls is performed through machine learning using theKaoKore dataset
Automatic recognition results can be used to support
construction of further datasets by using “CoordinateInput" function of the IIIF Curation Viewer*Reference
Alexis Mermet, Asanobu Kitamoto, Chikahiko Suzuki, Akira
Takagishi. "Face Detection on Pre-modern Japanese Artworks using
R-CNN and Image Patching for Semi-Automatic Annotation".
Proceedings of the 2nd Workshop on Structuring and Understandingof Multimedia heritage Contents (SUMAC'20), 2020, p.23–31,
doi:10.1145/3423323.3423412

A rectangle can be selected byentering numerical values for thebox displayed at "Coordinate Input"

All functions of the IIIF Curation Platform

15. KaoKore and art history research ★★
In joint research with Akira TAKAGISHI (Associate Professor, Departmentof Art History, University of Tokyo), We use all functions of the IIIF
Curation Platform●●●●

Development of a new KaoKore for the Yugyo Shounin Engi-Emaki,“Shojo-Kouji Kouhon” 『遊行上人縁起絵巻 清浄光寺甲本』
Picking out all Buddhist monks and nuns with the same clothing andhairstyles from Kouhon
we organized those facial expressions using the IIIF Curation Board,and exhaustively checking whether there are differences betweenmonks and nuns
We can also clarify differences in style of the artists*Reference
Chikahiko SUZUKI, Akira TAKAGISHI, Jun HOMMA, Alexis Mermet,Asamoto KITANOBU, "Difference between male and female in medieval
Japanese picture scrolls -Style comparative study for “Yugyo Shounin
Engi-Emaki” using GM Method-” (in Japanese), Proceedings ofJinmonkon 2020, pp.67-74, 2020

IIIF Curation Board
The IIIF Curation Board enables organizing and attachment of information tocurated images
http://codh.rois.ac.jp/software/iiif-curation-board/
There are various functions, so I will present them in stages
Start stage

Final stage

IIIF Curation Board

16. Reading in images ★
We can load a curation into theIIIF Curation Board

Start from ICManager

If you login and create acuration, you can easily startfrom ICManager
You can also start by directly
designating a curation URL or
simply drag and drop

You can also start by directly designating acuration URL, and dragging & dropping it to theICBoard

IIIF Curation Board

17. Moving and organizing images ★
Organization on a plane with theICBoard
Loaded images can be freely
moved on the plane by dragging
Images are organized as in the KJ
methodGrouping of facial expressions bymoving on the planeIIIF Curation Board

18. Automatic alignment and addition offrames ★

ICBoard has Assists groupingfunctions
Using automatic alignment andarranging boundary lines,components of image groups areeasier to understandRange selection and automatic alignment

Clarification of groups using boundary lines

IIIF Curation Board

19.Sticker function★

Attribute display using coloredstickers
Stickers can be attached toimages to clarify group attributes,etc. (5 colors)*Multiple stickers can be attached
to a single image

IIIF Curation Board

20. Note function★

Adds text information
Explanations based on
textual information can beadded by using the notefunction
Stickers can also beattached to notes, and theintention of color stickers
can be clarified

IIIF Curation Board

21. Display switching★

Turns information display on and off
Information display can be switched on and off
by pressing the information display (i) button,
・Information based on manifest
・Stickers
・Curation metadata
It is convenient to use this for various purposes.
For example, we can organize images with simple
display. On the other hand, indicating results step
we can show all information.

All functions of the IIIF Curation Platform

22. Converting humanities materials to microcontent★★

This is an initiative to extract parts of digitized humanities materials and reorganize them at a specific
granularity to enable use in research, shown in the practical example of KaoKore.
Workflow using the IIIF Curation Platform
Materials digitizedaccording to IIIF

Extraction and collectionof parts from materials

Adding metadata

Cross-search, listing,use in research

Conversion to microcontents: IIIF Curation Viewer/Manager/Editor
Analyze microcontes: IIIF Curation Finder/Board

There are various other examples beside KaoKore● "Edo Shopping Guide" http://codh.rois.ac.jp/edo-shops/● "Edo Tourism Guide" http://codh.rois.ac.jp/edo-spots/

Group Work andCurathon using ICP

Curathon
A curathon, like an ideathon, is an event where people brainstorm ideas oncuration for a specific time, present those ideas in front of everyone, and sharenew discoveries and ideas about curation
We “CODH” have conducted the following two curathon events● Digital Humanities for Japanese Culture: Resources and Methods○

http://codh.rois.ac.jp/tutorial/dhsi-2019/● "Picture Scroll Seminar" Curathon for Faculty of Letters, University of Tokyo○

http://codh.rois.ac.jp/tutorial/04/

IIIF Curation Viewer / IIIF Curation Manager

23. Conducting a curathon★

Curathon process

1. Lecture for participants on basic methods ofusing the IIIF Curation Platform

2. Participants freely perform curation

Curation focusing on hair buns

3. Participants share and comment on thecurations they have made*If organizer prepared beforehand, curathon go
more smoothly. Such as theme is decided
beforehand or database, dataset, and search
keywords are prepared.

Curation focusing on background plants

IIIF Curation Manager

24. Account sharing★

How to manage and share curathon
results
It is convenient to use a shared email
address and password for login to ICP
prepared beforehand by the organizers,
especially if you have a lot of
participants in the curathion or want toshare the results later.

IIIF Curation Editor

25. Filling in "description"
Adding explanatory text to curated images
Text can be added to curated images by editing the “description” item in the IIIF Curation
EditorIn a curathon, a record can be kept of the intent behind production of a curation, also thisfunction can be used like as explanation of picture in the exhibition

IIIF Curation Player

26. Display of "description”★

Display of explanatory text attached toa curation
The text in the description can bedisplayed with IIIF Curation Player.
Aside from presentation through a slide
show, this can also be used like
exhibition commentary by employing
the automatic playback function

The automatic playback button is displayed if
the IIIF Curation Player is set to full-screen
display.

Group work using the ICFinder
As one part of art history education, beginning
students perform group work as a way of reexperiencing the research process of theirpredecessors

All faces from the Ishiyama-dera engi emaki areprepared and students consider and discuss
which of the given faces were used to identify
styles in previous research.
Facial expressions are discovered using image
search of the IIIF Curation Finder
Masahiko AIZAWA, "Ishiyama-dera engi yokai," Ishiyamadera engi emaki shusei, Chuo-Koron Bijutsu Shuppan (2016)

All functions of the IIIF Curation Platform

27. Art history group work★★

Preparation for group work using the ICFinder

1. This time, we want to use a limited KaoKore(IIIF Curation Finder). So new ICP is preparedin my own server

2. I perfume curation of all faces in the subject
materials (in this case the 5 scrolls of theIshiyama-dera engi emaki)

3. Metadata is added for searching
4. Incorporation into ICFinder

IIIF Curation Finder

28. Art history group work★

Conducting group work using theICFinder
1. Participants consider the correct
image by using the search function ofthe ICFinder
2. They create derived curations, andmake presentations by each group
Participants search using attributes like direction and sex,
and choose faces determined to be of a "similar type" bytheir predecessors

ICP UseEnvironmentFor those who want to use the IIIF Curation Platform

123

Use case

Installation method

Project-specificenvironment

When using the demo environment
provided by CODH

No installation necessary

Not possible

Using the ICP client only

It is enough to download
the ZIP file, and expand
at the position where a
web server is needed

Not appropriate

ICP server is needed

Independent installation is
also possible, but it isconvenient to use theDocker

Possible

I'll explain 1 and 3. These are versatile methods.

All functions of the IIIF Curation Platform

29. Use in the demo environment★

You can use ICP in the demo environment provided by CODH
A demo environment, except for the IIIF Curation Finder, is available at the CODH
website and can be freely used
If you perform authenticated login, you can use curations across various functions
http://codh.rois.ac.jp/icp/
IIIF Curation Viewer： http://codh.rois.ac.jp/software/iiif-curation-viewer/demo/
IIIF Curation Manager： http://codh.rois.ac.jp/software/iiif-curation-manager/demo/
IIIF Curation Editor： http://codh.rois.ac.jp/software/iiif-curation-editor/demo/
IIIF Curation Player： http://codh.rois.ac.jp/software/iiif-curation-player/demo/
IIIF Curation Board： http://codh.rois.ac.jp/software/iiif-curation-board/demo/All functions of the IIIF Curation Platform (Docker version)

30. Use employing Docker★★

Docker version IIIF Curation Platform
http://codh.rois.ac.jp/software/icp-docker/
Easy construction way of your own environment
Convenient when you want create search function with the IIIF Curation Finder foryour own curations
Requires server management knowledge on the level of understanding port andproxy setting
Also requires basic knowledge of Docker
Nevertheless, beginners who understand the basics can handle it (I (Suzuki) am also
at that level)

In conclusion: ICP tutorials
An introduction to basic usage
of ICP, including procedures
presented in these recipes, areprovided this pagehttps://www.chsuzuki.com/icpt/
ICP tutorial is always being
updated, so please be sure touse itThank you for listening.
I would like to thank everyone who provided IIIF images and collaborated withthis research.