# SWHID specification kickoff meeting

Roberto Di Cosmo

► To cite this version:

Roberto Di Cosmo. SWHID specification kickoff meeting. Inria; Universite Paris Cite. 2023. hal-04121507

HAL Id: hal-04121507

https://hal.science/hal-04121507

Submitted on 7 Jun 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
SWHID specification
kickoff meeting

Roberto Di Cosmo

Director, Software Heritage
Inria and Université de Paris Cité

March 27th 2023

<figure><img src="image_2.png" /></figure>

Software Heritage
THE GREAT LIBRARY OF SOURCE CODE
<figure><img src="image_3.png" /></figure>

1. Introduction

2. Background on key concepts

3. The SWHID identifier

4. Turning SWHID into a publicly available specification

5. The road ahead
## Objective

bring you all up to speed on the SWHID specification effort, and kickstart the work

## Organization

* duration: 1 hour (45m plus 15m Q&A); write questions in the chat

* session will be recorded

* set your Zoom id to your full name

## Agenda

* recall basic notions of identifiers, then focus on the SWHID

* review of the way SWHIDs are composed and computed

* focus on the key parts of the specification that need work

* guided tour of the contribution and editorial process

* governance and licensing
1. Introduction

2. Background on key concepts

3. The SWHID identifier

4. Turning SWHID into a publicly available specification

5. The road ahead

<figure><img src="image_4.png" /></figure>
## Identification of a book

<figure><img src="image_5.png" /></figure>

Goal: identify a book

* one ISBN number per published book

* ISO 2108 standard specification

## Location of (a copy of) a book

<figure><img src="image_6.png" /></figure>

Goal: find (a copy of) a book

* many locations (locations can change!)

* many approaches for call numbers

we are interested in **identification**, not in location
In a nutshell

(for more info see this dedicated blog post)

Main difference: how the *relation* between *identifier* and *designated object* is created and maintained. *Persistence* is a key desired property.

<table><thead><tr><th></th><th>Extrinsic</th><th>Intrinsic</th></tr></thead><tbody><tr><th>relation</th><td>register</td><td>convention</td></tr><tr><th>persistence</th><td>external<sup>a</sup></td><td>internal</td></tr><tr><th>pre-internet</th><td>passport number, ISBN, SSN, etc.</td><td>Music/Chemistry notations<br>e.g. NaCl is table salt</td></tr><tr><th>internet era</th><td>DOI, Handle, Ark, etc.</td><td>cryptographic hashes<br>e.g.: git, bitcoin, SWHID</td></tr></tbody></table>

Here we are interested in normalising the SWHID intrinsic identifier

<footnote><sup>a</sup>"persistence... is a function of administrative care" RFC 3650 (Handle System Overview, 2003)</footnote>
1. Introduction

2. Background on key concepts

3. The SWHID identifier

4. Turning SWHID into a publicly available specification

5. The road ahead

<figure><img src="image_7.png" /></figure>
## Structure of a SWHID identifier

link to full docs

<figure><img src="image_8.png" /></figure>

## Current status

30+B SWHIDs in the Software Heritage archive

Mention in Linux Foundation's SPDX 2.2; IANA registered; WikiData P6138
<figure><img src="image_9.png" /></figure>
Contents

GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007

Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>.
Everyone is permitted to copy and distribute verbatim Copies
of this License document, but changing it is not allowed.

Prepared

The GNU General Public License is a free, copy-left license for
software and other kinds of works.

The licenses for most software and other practical works are designed
to take away your freedom to share and charge the works. By contrast,
the GNU General Public License is intended to guarantee your freedom
to share and change all versions of a program, to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.

When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you can recover source code or can get it if you
want it; that you can charge the software or use pieces of it for free
programs, and that you know you can do so.

...To protect your rights, you need to p...

sha1: 8624bcdae55baeaf...

sha256: 8ceb4b9ee5aded...

sha1_git: 94a9ed024d385...
length: 35147
<figure><img src="image_10.png" /></figure>
## Directories

?

gitignore

AUTHORS

LICENSE

MANIFEST.in

Makefile

Makefile.local

README.db testing

README.dev

bin

debian

docs

requirements.txt

setup.py

sql

swi

utils
Directories

gtgignore

AUTHORS

LICENSE

MANIFEST.in

Makefile

Makefile.local

README_db_testing

README.dev

bin

debian

docs

requirements.txt

setup.py

sql

swh

utils

100644 blob c5baade4c44766042186e7f858c0fd63d587ebf09 .gitignore

100644 blob 2d8a34af6f52cf3cf6b0c2f7bd6848fbd255e77f AUTHORS

100644 blob 94a9ed024d3859793618152ea559a168bbccbb5e2 LICENSE

100644 blob d9b2665a435a43f8a79a84e8867751dfb95c7bb MANIFEST.in

100644 blob 524175c2bad0b35b975f79284c2f5a6d5eaf2e4b Makefile

100644 blob 5c7e3a5bddb038682ba7793f440492ed9677bb3 Makefile.local

100644 blob 8617980629cd24e6080404f09aa749b085b3e070 README.db_testing

100644 blob 76b29f48f715e0869c414d38d78d7ce08e5c154b README.dev

040000 tree e1e10ecef948af0b93adb0372afc89f12e92618a bin

040000 tree 83e56d0bea7793c77a45a345c80fcb8a5f503013 debian

040000 tree a34c9c4ba213f0ced67f9816348d27955577fa5 docs

100644 blob f2a6d32c6135aa7287bbd76167b01df2ae4f1539 requirements.txt

100755 blob eee147c36caf1bbcd28d0a8d026cb5b68188bc setup.py

040000 tree 224bb4c1f4c67fca1d160baffd2d06094e7e1abf3 sql

040000 tree 8631c9cd77bbe993168107ab5baf51f40c6300be swh

040000 tree 8fb905b56ba8ed692f1209b2773b474c6c1d66c1 utils

id: 515f00d44e92c65322aaa9bf3fa097c00ddb9c7d
<figure><img src="image_11.png" /></figure>
Revisions

Details

Changes

Files

SHA: 963634ca6ba5dc37e3ee426ba091092c267f9f6

Author: Nicolas Dandrimont <nicolas@dandrimont.eu> (Thu Sep 11 14:26:13 2016)

Committer: Nicolas Dandrimont <nicolas@dandrimont.eu> (Thu Sep 11 14:26:13 2016)

Subject: provenance/tasks: add the revision -> origin cache task

Parent: fc3a8b59ca1df424d860f2c29ab07fee4dc35d10 test_storage: properly pipeline origin and cont...
provenance tasks: add the revision -> origin cache task

swb/storage/provenance/tasks.py

77

<figure><img src="image_12.png" /></figure>

tree 515f00d44e92c65322aaa9bf3fa097c00ddb9c7d
parent fc3a8b59ca1df424d860f2c29ab07fee4dc35d10
author Nicolas Dandrimont <nicolas@dandrimont.eu> 1472732773 +0200
committer Nicolas Dandrimont <nicolas@dandrimont.eu> 1472732773 +0200

provenance/tasks: add the revision -> origin cache task

<figure><img src="image_13.png" /></figure>

id: 963634dca6ba5dc37e3ee426ba091092c267f9f6
<figure><img src="image_14.png" /></figure>
## Releases

object: c0c9f16be134f593e7567507a1761b156e6eb1d

type: commit

tag: v0.5.1

tagger: Nicolas Dandrimont <nicolas@dandrimont.eu> 1472042163 +0200

Release swh.storage v0.5.1

- Add new metadata column to origin

- Update swh-add-directory script for updated API

— BEGIN PGP SIGNATURE —

tag v0.5.1

Tagger: Nicolas Dandrimont <nicolas@dandrimont.eu>

Date: Wed Aug 24 14:36:03 2016 +0200

Release swh.storage v0.5.1

- Add new metadata column to origin, visit

- Update swh-add-directory script for updated API
  [—]

commit c0c9f16be134f593e7567507a1761b156e6eb1d

IQUBAABCAABdQXjZVTNFhuuWw+bGf+QGRhbmRvaWlvbnQuZ2XUACs+Q7ALWMo2+
neworq/a050505Qe+WN3xgys+K1KEVH1wWKaB6xj+TAX2EILDHtuf
ahpZ6p308nq6aC1+YxrBch3LYmZtWKwY8rBWN4kE5yD8oaPhwhBAD512
ICB2uQxCrD9rKWPw2Xg+hd0wWny15D0rW72K4mUy0pHYP55y0
IGEndWnoV7wH1Vm61n50b7SmiRxaA+bcced0uT2Zj+liuJqC8cygN3n3nfL
qsJmUkxy2j8HgH1pv+50wbin55THr0qoVgPKHSPF9UdHZHkCa0
tj6kAwuyUy0M0b+KvJELbR3+yWBJ3p0s1aV9b0TH6E1ALNHNDEaKoKM
drgMRaxl1l1g0EDn5rW7G6sDkKPPhngfLVQ3nV3gAQQTnr1RpM0269IwA9WC
Gk1HT4hH46wW72ye5UvXGFvVV9y0Q42R/wn+0dZdc4drj+UJ0Mn
R0TTUbxUxK0CgkX5Y7Tnp1g0C75Tb0Da64Z21k1m0kGwCVrQh9n
nnhBB5HBmNqoyFyD5T0pU5Y7k7pYRRUGXDeRk0CsWk+WKUZGixy6yQj09
gwgZ0j50WCB000denAL2+HvPraVykMeJUg6c29+EHvIuK=
=0kxP
— END PGP SIGNATURE —

id: 85083a5cc14a441c89dea73f5bdf67c3f9c6c16fb
<figure><img src="image_15.png" /></figure>
Snapshots

commit 8Bff8b257709525e63c2169146c53a1d9158 refs/heads/atime
commit ba543a24e3f9f3234a62c29c4cd63c67eb refs/heads/directory-listing-arrays
commit d69e0df8b238f365890b27fe1c05627238dc5c refs/heads/foo
commit c7f7f9ee6a82b2f849690f5a0196f67d468e88 refs/heads/easter
commit 7ecca197fcd66202048e5a6b1e8b43b6a82c2 refs/heads/tmp-directory-add
commit 642a20573d0858a85d4272d53e4f8b252e82e refs/heads/tmp-generic-releases
tag 20f843b3137f7c8d966597799f4907c757755 refs/tags/v0.1
tag 72a219931a38d539996db8667fb0bee72aee2cd refs/tags/v0.10
tag 35980c8a06b07e58376705a300b4fa55cc refs/tags/v0.11
tag 333784274e48b3ab69a6777d5856f6674c6556 refs/tags/v0.12
tag 06f7465755b372f759811c2b3a636f3c4b35d5 refs/tags/v0.13
tag 5a63757e6a8b05b85d4744267692a11e323bd refs/tags/v0.14
tag 586b8fa4588b4f8ab5f159396743cb41a97c refs/tags/v0.15
tag 8cd88854f988b3f3617742d2096f68a6e5lc refs/tags/v0.16
tag 5d42444ae3f8bed35efb202fee35c8989bc16d refs/tags/v0.17
tag 2282ra135d0812222e55659462e106c4993d9 refs/tags/v0.18
tag 06978a5ca8d54977f8d4d4a808ce2636ef47c refs/tags/v0.19
tag 32b85a5c9f2a323aa6d5f13a6d5382e275c7567 refs/tags/v0.2
tag 3147c361c1e46d4629481b88d1237abff7c27 refs/tags/v0.20
tag 215e5e06ab11080e28e72e6b6d7378908 refs/tags/v0.21
tag 317b68c2072a50625212475a1e5df05ffafd refs/tags/v0.22
tag 8cb bee80a47d3fc5d262789e468a16ac72a80c refs/tags/v0.23
...

git show-refs

id: b464cad1b66ff266a37b46ea6e7a04b545e904b
1. Introduction

2. Background on key concepts

3. The SWHID identifier

4. Turning SWHID into a publicly available specification

5. The road ahead

<figure><img src="image_16.png" /></figure>
Origin of the SWHID acronym

SWHIDs were born as

* intrinsic identifiers

* designed for

* software source code

* archived in Software Heritage

Hence the acronym:

SW Software

H Heritage

ID IDentifier

Proposed reformulation

SWHIDs are

* based on a cryptographic hash

* can be used independently of the Software Heritage Archive,

* not restricted to source code

Hence the proposal:

SW Software

H Heritage Hash

ID IDentifier
## Our goal

Create a specification that

* is complete, precise and non ambiguous (pictures are simplified representations!)

* allows any "person skilled in the art" to implement the same calculation algorithm

To this end we need to get right:

* five key parts in the core: cnt, dir, rel, rev, snp

* qualifiers: easier, but important too

* reference to external standards used (e.g. SHA algorithm)

## What we have

* high level documentation from Software Heritage

* two complete implementations (**one in Python** for SWH, **one in OCaml** for Opam)

* a draft specification that needs to be completed
## Bylaws

Detailed in the **Governance Document**

* consensus based decision making

* open process to produce an **Approved Specification**

## License

**specification** Community Specification License

**contributions** specific CLA accepted by sign-off on GitHub

## Coordination by the **Core Team**

Alexios Zavras, Jean-François Abramatic, Morane Gruenpeter, Roberto Di Cosmo, Stefano Zacchiroli

Roles: **maintainers**, editors
Specification document

sources GitHub repository https://github.com/swhid/specification
rendered swhid.org website

Working on the specification

* **contribution workflow** based on **issues** and **pull requests** on GitHub

* N.B: pull requests **must be signed-off**

* editors **merge pull requests** on **consensus**

* merges **rebuild** the **rendered version** on swhid.org

Mailing list: swhid-discuss

used mostly for coordination by the editors

(e.g. scheduling meeting, votes, version freeze, etc.)
<figure><img src="image_17.png" /></figure>

## Working with issues and pull requests

* the directory SWHID

* issue

* pull request

## Taking a vote

* the name of the game poll is open at https://bit.ly/swhid-name-poll
1. Introduction

2. Background on key concepts

3. The SWHID identifier

4. Turning SWHID into a publicly available specification

5. The road ahead

<figure><img src="image_18.png" /></figure>
## Timeline

* phase 1 complete and accurate v1.0 as soon as possible (end of April desirable)

* help establish the whole process

* cover *only* what is already known and being used

* focus on items labeled **blocker**, make them *complete, precise* and *non ambiguous*

* phase 2 work towards v1.1

* handle other feedback / input / requests

* get the version candidate to become an ISO standard

## Questions?

## Links

* main entry point: https://swhid.org

* specification sources: https://github.com/swhid/specification