<figure><img src="image_1.png" /></figure>

HAL
open science

# MemoRekall-IIIF: An open source and versatile web application for video and digital document annotation

Jacob Hart, David Rouquet, Clarisse Bardiot

► To cite this version:

Jacob Hart, David Rouquet, Clarisse Bardiot. MemoRekall-IIIF: An open source and versatile web application for video and digital document annotation. ADHO Digital Humanities Conference 2023 (DH2023), Jul 2023, Graz, Austria. , Digital Humanities 2023: Book of Abstracts, 2023, 10.5281/zen-odo.7961822 . hal-04172101

HAL Id: hal-04172101

https://hal.science/hal-04172101

Submitted on 27 Jul 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
<figure><img src="image_2.png" /></figure>

# MemoRekall - IIIF
## An open source and versatile web application for video and digital document annotation

Jacob HART, David ROUQUET, Clarisse BARDIOT

<figure><img src="image_3.png" /></figure>

www.memorekall.com

* Researchers need tools to enrich **digital** or **digitized** documents, both to **support the analytical process** and to render the results of their work.

* We need to **expand on the original content** by **annotating** it and **linking** it to a multimedia context (through manual and AI/ML processes).

* Existing tools differ regarding their supported formats, annotation modalities, cooperation features, sharing capabilities, interoperability, extensibility, etc.

* Users find themselves in a situation of **vendor lock-in**. Work is 'trapped' in **proprietary formats**, leading to the **fragmentation of annotated content** in silos.

* As such, we propose a prototype, MemoRekall-IIIF: an environment that focuses on **extensibility** and **interoperability** to allow for **collaborative** and **incremental** research.

* We are actively developing the tool, and are eager to get **user feedback**. Test out the demos, and let us know what you think.

## Prototype Development: IIIF Integration

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

MemoRekall-IIIF is a complete re-implementation of MemoRekall - a video annotation tool developed in a co-creation process between **developers** and **digital humanities researchers**. It follows the same practice-led development process, but presents a number of **technical** and **epistemological shifts**.

Integration of IIIF (International Image Interoperability Framework). Allows for **interoperability**, **sharing** and **cumulative work** that builds on **existing GLAM databases**. MemoRekall-IIIF is based on the widely-used IIIF viewer **Mirador**.

We have **extended** the Mirador viewer:

* Full **video** and **audio** support.

* Interactive annotations.

* Fluid multimodal document network navigation.

* Data-driven interfaces.

## MemoRekall to MemoRekall-IIIF

<table><thead><tr><th></th><th>MemoRekall</th><th>MemoRekall-IIIF Prototype</th></tr></thead><tbody><tr><td>Open Source</td><td>✓<br>www.gitlab.tetras-libre.fr/rekall/</td><td>✓<br>www.gitlab.tetras-libre.fr/iiif/POC-mirador</td></tr><tr><td>Sharing Capabilities</td><td>✓<br>Direct link or embeddable code.</td><td>✓<br>Direct link or embeddable code.</td></tr><tr><td>Cooperation Features</td><td>✓<br>Several users can work on a capsule.</td><td>✓<br>Several users can work on a capsule.<br>+ Adoption of a git-inspired version control model (versions, branches etc.).</td></tr><tr><td>Modularity and Extensibility</td><td>X<br>Open source but difficult to extend.</td><td>✓<br>Implemented as a fork of the Mirador IIIF viewer: inherently modular.<br>+ Plugin author-friendly design.</td></tr><tr><td>Interoperability</td><td>X<br>XML file format (non-standard).</td><td>✓<br>Based on the IIIF standard.</td></tr><tr><td>Supported Formats</td><td>≈<br>Support for small files, but the main video is hosted on YouTube or Vimeo.</td><td>✓<br>Same features + An instance of Peertube for large video files.<br>+ IIIF: refer to any online-hosted media.</td></tr><tr><td>Perspective Fluidity</td><td>≈<br>Generally locked into focusing on one principal video resource.</td><td>✓<br>Same perspective possible + Shift towards a document-network perspective.</td></tr><tr><td>Annotation Modalities</td><td>≈<br>Wide range of annotation modalities that can be attached to various media and hold metadata.</td><td>✓<br>Opens the annotation modalities further: annotations can be atemporal; metadata uses Dublin Core standard; annotations can be linked to another IIIF manifest.</td></tr><tr><td>Data-Driven Interfaces</td><td>X<br>There is no conception of data-driven interfaces.</td><td>✓<br>Create data-driven interfaces for the navigation of collections of IIIF manifests.</td></tr></tbody></table>