# Ontology-based Web tools for retrieving photogrammetric cultural heritage models

M Ben Ellefi, P Drap, O Papini, D Merad, J Royer, M Nawaf, E Nocerino, K Hyttinen, Jean-Christophe Sourisseau, T Gambin, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

M Ben Ellefi, P Drap, O Papini, D Merad, J Royer, et al.. Ontology-based Web tools for retrieving photogrammetric cultural heritage models. ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2019, XLII-2/W10 (XLII-2/W10), pp.31 - 38. 10.5194/isprs-archives-xlii-2-w10-31-2019 . hal-03597510

HAL Id: hal-03597510

https://amu.hal.science/hal-03597510

Submitted on 4 Mar 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
ONTOLOGY-BASED WEB TOOLS FOR RETRIEVING PHOTOGRAMMETRIC
CULTURAL HERITAGE MODELS

M. Ben Ellefi<fnref n="1" />, *, P. Drap<fnref n="1" />, *, O. Papini<fnref n="1" />, D. Merad<fnref n="1" />, J. P. Royer<fnref n="1" />, M. M. Nawaf<fnref n="1" />, E. Nocerino<fnref n="1" />, K. Hyttinen<fnref n="2" />, J. C. Sourisseau<fnref n="3" />,
T. Gambin<fnref n="4" />, F. Castro<fnref n="5" />

Commission II, WG II/9

KEY WORDS: Ontologies, Photogrammetry, Semantic Web, Cultural Heritage, Domain Knowledge

ABSTRACT:

A key challenge in cultural heritage (CH) sites visualization is to provide models and tools that effectively integrate the content of a CH data with domain-specific knowledge so that the users can query, interpret and consume the visualized information. Moreover, it is important that the intelligent visualization systems are interoperable in the semantic web environment and thus, capable of establishing a methodology to acquire, integrate, analyze, generate and share numeric contents and associated knowledge in human and machine-readable Web. In this paper, we present a model, a methodology and a software Web-tools that support the coupling of the 2D/3D Web representation with the knowledge graph database of *Xlendi* shipwreck. The Web visualization tools and the knowledge-based techniques are married into a photogrammetry driven ontological model while at the same time, user-friendly web tools for querying and semantic consumption of the shipwreck information are introduced.

# 1. INTRODUCTION

Studies on Underwater Cultural Heritage (UCH) sites induce
the use of computerized techniques to handle, preserve and anal-
yse the information produced. For complex archaeological sites,
such as deep underwater shipwrecks, there is a special need
for systems that support efficient storing and allow for a smart
querying of data evolving over time.

This work is centered on the *Xlendi* shipwreck, named after the
place where it was found off the Gozo coast in Malta. The ship-
wreck was located by the Aurora Trust, an expert in deep-sea
inspection systems, during a survey campaigns in 2009. The
shipwreck is located near a coastline known for its limestone
cliffs that plunge into the sea and whose foundation rests on
a continental shelf at an average depth of 100 m below sea
level. The shipwreck itself is therefore exceptional; first due
to its configuration and its state of preservation which is partic-
ularly well-suited for our experimental 3D modelling project.
The examination of the first layer of amphorae also reveals a
mixed cargo, consisting of items from Western Phoenicia and
Tyrrhenian-style containers which are both well-matched with
the period situated between the end of the VIII and the first half
of the VII centuries BC. The historical interest of this wreck,
highlighted by our work, which is the first to be performed on
this site, creates a real added-value in terms of innovation and
the international reputation of the project. A photogrammetric
modelling of *Xlendi* shipwreck, done in 2014, was presented
in a previous work (Drap et al., 2015) in the framework of
the GROPLAN project<fnref n="1" />. The University of Malta have already

organized photogrammetric campaigns over the years and start-
ing archaeological excavation. We have now five years survey
to observe the site change over the time.

In 2001, the UNESCO Convention for the Underwater Cultural
Heritage established the necessity of making all data publicly
available on URL<fnref n="2" />. This work is fully built upon this philoso-
phy, proposes a way to share and analyze archaeological data.
Indeed the documentation of underwater sites is becoming a
growing concern and a large amount of research lab is focus-
ing on this topic (Scaradozzi et al., 2013), (Scaradozzi et al.,
2017). In that context, CH sites are tending to allow data ex-
change and accessibility in a worldwide network scale instead
of single isolated database. Note here that relational data base
systems tend to be rigid especially when it concerns the schema
modifications and extensions. This particular feature does not
meet our expectations where the scheme of historical sites need
to be often updated as new CH discoveries are made, i.e, new
typologies can be often discovered as new dives are made in
*Xlendi*. In our view, the key factor to achieve a semantic ex-
ploitation of a shared open CH data consists of constructing a
dynamic system that supports schema flexibility, data accessi-
bility and knowledge sharing. Toward the development of CH
applications that provide multimedia access to distributed col-
lections of CH resource, we cite recent projects that among oth-
ers, ADS<fnref n="3" />, ARIADNE<fnref n="4" />, EUROPEANA<fnref n="5" /> and STITCH<fnref n="6" />.

The main objective of this work is the development of an in-

<footnote n="1">¹ Aix Marseille University, CNRS, ENSAM, Université De Toulon, LIS UMR 7020, 13397 Marseille, France</footnote>

<footnote n="2">² University of Dundee, Perth Rd, Dundee, Scotland UK;</footnote>

<footnote n="3">³ Aix Marseille University, CNRS, Ministère de la Culture et de la Communication, CCJ UMR 7299, 13094 Aix En Provence, France;</footnote>

<footnote n="4">⁴ Archaeology Centre, University of Malta, Msida MSD 2080, Malta;</footnote>

<footnote n="5">⁵ Ship Reconstruction Laboratory 4352 TAMU, Texas A & M University, College Station, Texas 77843, USA.</footnote>

<footnote>*Corresponding authors: {mohamed.ben-ellefi, pierre.drap} @univ-
amu.fr</footnote>

<footnote n="1">¹http://www.groplan.eu</footnote>

<footnote n="2">²http://www.unesco.org/new/en/culture/themes/
underwater-cultural-heritage/</footnote>

<footnote n="3">³http://data.archaeologydataservice.ac.uk/query/</footnote>

<footnote n="4">⁴http://www.ariadne-infrastructure.eu/</footnote>

<footnote n="5">⁵https://www.europeana.eu/portal/fr</footnote>

<footnote n="6">⁶https://www.cs.vu.nl/STITCH/</footnote>
formation system capable of establishing a methodology to acquire, integrate, analyze, generate and share numeric contents and associated knowledge in a standard machine readable format. In this paper, we present a knowledge graph based Web tools, where semantic data are stored in a graph database and schematized by an explicit specification of a conceptualization-ontologies (Doerr, 2009) (Guarino et al., 2009). The graph structure meets our expectation of a **schema flexibility** since data can be separated from the schema model. In this way, the schema modification can be performed without changing the data instances or worrying about foreign keys, nested queries, and complex joins. Hence, new data can be inserted into the existing graph without loss of application functionality.

Ontologies have a meaningful role to play in supporting **knowledge sharing** expectations, as it provides a shared and common understanding of data and, in some cases, services and processes that exist within a domain (in our case, photogrammetry). Such common understanding facilitates communication between people and information systems (machines) and an enhanced ability to search for information across different knowledge repositories. This conceptualization allows the coupling of concepts within an ontology to information and processes within the organization being represented. In fact, ontologies allow application designers to fully understand the meaning and context of the information being modelled. Such common understanding is supported by the Linked Open Data (LOD) structure which, following semantic web best practices (Lóscio et al., 2017), (Auer et al., 2013) and the Linked Data principles<fnref n="7" />, will meet our expectations for **data accessibility** via the use of URIs (W3C Uniform Resource Identifier) to identify resources and allow people (and machines) to look them up on the web.

LOD knowledge graphs can be represented by a set of connected together triples (subject-predicate-object) defined by W3C for RDF<fnref n="8" /> stored within a triple store, e.g. DBpedia (Auer et al., 2007) knowledge graph bases which has been constructed using structured sources such as Wikipedia infoboxes. Such triples structure expresses information by defining relationships between data objects which allow resource to be identified, disambiguated and meaningfully interlinked. For example, encoding the fact that "Amphorae is an Artifact" in the knowledge graph is done by storing two vertices, one for "Amphorae" and one for "Artifact", with a directed edge originating from Amphorae and pointing to Artifact of type "isA". Moreover, this infrastructure will play a major role in providing a reasoning functionality where, the navigating between individual entities (near one another) in a graph is more obvious than trying to aggregate all attributes related to a given entity from different tables (as in a relational database). For example, the question "Is amphorae an artifact?" can be answered by simply walking through the graph, starting at "Amphorae" and walking to "Artifact", testing edges and concepts along the way. Hence, data can be easily consumed and combined with other linked data.

In the context of *Xlendi* shipwreck, we adopted knowledge graph techniques to represent the photogrammetrical information of the site where data are stored within a triple store and schematized by a photogrammetry driven ontology *Arpenteur*<fnref n="9" />. This ontology has been developed in a collaborative work between archaeologists, ontology designers and surveyors (Drap et al.,

2015). *Arpenteur* ontology conceptualizes the whole photogrammetry process and involved CH artifacts in terms of their typologies, photogrammetric measurement and spatial representation, as depicted in (Ben Ellefi et al., 2018b).

The paper is organized as follows: Section 2 presents the surveying process of *Xlendi* shipwreck by photogrammetry. The storage system architecture of *Xlendi* semantic datasets is depicted in Section 3. The web tools are presented in Section 4 where: (i) Section 4.1 details the interactive queries performed in back-end of the web-tools with a focus on what ontologies can do and describing how they can be used in support of archaeological research; and (ii) Section 4.2 presents our implementation of 2D/3D web tools where an example use case is detailed. Finally, Section 5 concludes and presents future work plans.

## 2. PHOTOGRAMMETRY

Five photogrammetric surveys were done from 2009 to 2018. We had the opportunity to improve the method and try new experiments over the time. The first one, in 2009, was only a recognition photographic campaign done to identify the wreck. We process the 80 images later and use the others survey to scale and orient this one.

The second survey was done in 2014, thanks to the collaborative project GROPLAN, by COMEX with CNRS LIS and CCJ. We used the COMEX submarine Remora 2000 and a trifocal system used to acquire synchronized images on site. This approach allowed us to work safely at this depth and the trifocal system gives us a proper scale without intervention on the site. A huge amount of images were obtained and elaborated with the Photoscan software, which was used for the following campaigns as well. This campaign was also used to develop and test new known object extraction approaches in a site in an attempt to automatically identify and measure known artifacts on the site (amphorae and grinding stone). Nevertheless, several problems were still present. The overall cost of the operation remained high, we did not have a precise reference to the vertical and even if some surface artifacts could be lifted by ROV we could not consider an archaeological excavation.

The successive campaigns, 2016, 2017 and 2018, were organized by the University of Malta, with the participation of University of Dundee, CNRS LIS and CCJ. The idea was to perform the survey with a team of professional divers and archaeologist using rebreather. These three successive campaigns allowed the University of Malta to implement solutions to the problems identified during the first campaign. Scuba divers have drastically reduced the cost of the mission over the flexibility offered by a team of four professional divers has allowed to establish a method of shooting photographs with excellent light management leading to excellent recordings.

Over the years, the survey method has been refined. Timmy Gambin and his team built up procedures and tools to organize excavation and survey by divers at this depth. Cement dead bodies with coded targets were placed around the wreckage to ensure continuity of the reference system over the dive, a stable tripod with a bubble level and coded targets was installed to give a reference to the vertical.

Finally the last year in 2018 a real archaeological excavation was carried out at 110 meters depth with a water suction (with

<footnote n="7">⁷https://www.w3.org/DesignIssues/LinkedData.html</footnote>

<footnote n="8">⁸Resource Description Framework: https://www.w3.org/RDF</footnote>

<footnote n="9">⁹http://arpenteur.org/ontology/Arpenteur.owl</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 1: Shooting images: one diver for the camera and two other for the light. Photo: Dave Gration.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Figure 2: The bubble tripod and two cement blocks.</figcaption></figure>

pump immersed at 20 meters), 2x2 meters quadrat, photogram-
metric monitoring, removal of artifact and appearance of lower
layers.

<figure><img src="image_5.png" /><figcaption>Figure 3: Excavation on *Xlendi* by divers with rebreather at 110 meters depth. Photo: Dave Gration.</figcaption></figure>

A rigorous approach allowed us to obtain consistent records
documenting sampling over the years (2009 - 2018) and ex-
cavations completed in 2018 (Gambin et al., 2018). All of these
surveys are now available in 2D or 3D on the project website
and formalized by an ontology.

This ontology takes into account the manufactured items sur-
veyed, as well as the method used to measure them, in this
case, photogrammetry (photogrammetric data as oriented pho-
tographs, cameras, 3D points and their projections as well as
camera distortion and precision estimator).

The surveyed items are therefore represented from the measure-

ment point of view and has access to all the photogrammetric
data that contributed to its measurement in space.

A 2D and 3D web interface, accessible on the project web site,
allows access to all these data and semantic queries through
them and over the web.

3. XLENDI SEMANTIC DATASETS

The University of Malta has been surveying the *Xlendi* site since 2009 and the dataset has been enriched by data collected over time, as detailed in Section 2. These successive temporal states of the same site required the development of a management and query system able to handle data according to a spatio-temporal context. In order to implement such query system, we opted for a full materialization strategy to store the different temporal versions of the datasets. Using this strategy, all different versions of the evolving dataset are stored explicitly in separate datasets within the triple store. This storage strategy will avoid any pre-processing cost in order to store the new versions and also ensure that the query processing over versions is usually efficient as all the versions are already materialized in the archive.

As a knowledge base storage system, we used the Apache Jena
Fuseki<fnref n="10" /> component which provides a SPARQL server that im-
plement Apache TDB<fnref n="11" /> triple store for persistent storage and
provides the SPARQL protocols for query, update and REST
update over HTTP.

In this triple store system, we have separated the dataset that
contains *Xlendi* artifacts descriptions from the temporal datasets
which contains photogrammetrical information corresponding
to a given excavation.

**3.1 Xlendi artifact Dataset**

The artifact dataset contains information about the typological
dimension of artifacts and the 3D **theoretical** model descrip-
tion: the set of 3D points describing the model, the rotation
matrix and translation to align the multi-temporal models to
a common coordinate reference system. The dataset is pub-
lished as Linked Open Data (LOD) following the semantic web
best practices (Lóscio et al., 2017), (Auer et al., 2013) and the
the Linked Data principles (coined by Tim Berners-Lee)<fnref n="12" />: (1) URIs to identify the things in your data, (2) 'HTTP://' URIs so people (and machines) can look them up on the web, (3) When a URI is looked up, return a description of the thing in the W3C Semantic Web Format (typically RDF, RDF-Schema, OWL), (4) Include links to related things.

A YASGUI SPARQL client (Rietveld, Hoekstra, 2017), allow-
ing to query the *Xlendi* artifacts dataset, is made available on
URL<fnref n="13" />. *Xlendi* resources are modeled by the Arpenteur ontol-
ogy concepts: 'BoundingBox', 'GrindingStone', 'Item_Point3D',
'MeasuredPointManager', 'RotationMatrix' and 'Transformation_3D'.
In its current statue, the dataset contains 1,663,933
triples representing information about 75 amphorae and 55 grind-
ing stones and their photogrammetry descriptions. The ontolog-
ical model of this dataset is detailed in previous works (Ben Ellefi

<footnote n="10">¹⁰https://jena.apache.org/documentation/fuseki2/</footnote>

<footnote n="11">¹¹https://jena.apache.org/documentation/tdb/index.html</footnote>

<footnote n="12">¹²https://www.w3.org/DesignIssues/LinkedData.html</footnote>

<footnote n="13">¹³http://www.arpenteur.org/ontology/sparql.html</footnote>
et al., 2018b), (Ben Ellefi et al., 2018a). The Arpenteur ontology is aligned on the well-known CIDOC-CRM ontology already used in CH context (Niccolucci, 2017), (Niccolucci, Hermon, 2017) (Gaitanou et al., 2016). *Xlendi* artifact dataset is made available as open data on the datahub<fnref n="14" /> under the name 'Xlendi Amphorae'.

## 3.2 Xlendi Temporal Dataset

Successive dives on the *Xlendi* wreck have resulted in seven temporal datasets corresponding to the surveys dates listed in table 1. These temporal survey datasets describe the photogrammetric process in term of camera settings, interior and exterior orientation parameters, extracted and matched 2D points and their corresponding 3D points. We modeled *Xlendi* temporal resources by Arpenteur ontology on the following concepts: 'Transformation 3D', 'RotationMatrix', 'RadialDecenteringDistortion', 'Point_3D', 'Photograph', 'PhotoManager', 'MeasuredPointManager', 'ImagePointManager', 'ImagePoint', 'Item_Point3D', 'DigitalCamera' and 'BundleModel'.

We present in Table 1 a statistical report for each *Xlendi* temporal datasets by survey dates. The goal of this report is to highlight the quality of photogrammetrical information within each temporal datasets in term of the distribution of photographs, 2D/3D points and the total number of triples.

<table><caption>Table 1: The distribution of photographs, 2D/3D points and total number of triples for each of *Xlendi* temporal datasets</caption><thead><tr><th>Datasets</th><th>Photos</th><th>2D points</th><th>3D points</th><th>Triples</th></tr></thead><tbody><tr><td>2009-04-09</td><td>76</td><td>210493</td><td>88770</td><td>4906044</td></tr><tr><td>2014-07-18</td><td>3303</td><td>2102942</td><td>409255</td><td>38576785</td></tr><tr><td>2016-09-15</td><td>1232</td><td>773940</td><td>205579</td><td>15407574</td></tr><tr><td>2017-09-14</td><td>1370</td><td>1165167</td><td>396853</td><td>25099116</td></tr><tr><td>2018-09-04</td><td>1058</td><td>881424</td><td>311140</td><td>19228709</td></tr><tr><td>2018-09-13</td><td>1102</td><td>682414</td><td>251099</td><td>15122696</td></tr><tr><td>2018-09-21</td><td>968</td><td>886607</td><td>292203</td><td>18881094</td></tr></tbody></table>

Similar to the artifact dataset, seven SPARQL user interfaces, allowing to query temporal datasets, are made accessible online via our 2D/3D Web tools. For example, the SPARQL GUI for the 2018-09-21 dataset is available on URL<fnref n="15" />.

## 4. XLENDI WEB-BASED TOOLS

In this section, we describe our Web tools accessible via http://www.lsis.org/groplan/svg/xlendi/xlendi.html. We set up a set of predefined user-friendly queries, as detailed in Section 4.1, to exploit and analyze *Xlendi* knowledge database. The Web tools do not require specific knowledge of Semantic Web technologies and SPARQL query language to run the user-friendly queries. On the other hand, users with Semantic Web knowledge can perform advanced SPARQL queries over *Xlendi* datasets via the web client user interface YASGUI available online in our Web tools (described in Section 3).

### 4.1 Querying *Xlendi* Semantic Datasets

*Xlendi* semantic data are processable via SPARQL queries that are performed in the back-end of our Web tools and presented to the user as the following queries.

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

Xlendi wreck survey, from photogrammetry [2D - 3D - Timeline - Sparql]

<figure><img src="image_8.png" /></figure>

Artifact removed: green in 2014, red in 2016, and yellow in 2017

<caption>Figure 4: Xlendi Orthophoto drawing in our 2D Web tool (powered by Raphaël¹⁶)</caption>

* Select artifacts having the same typology: This user-friendly query corresponds to a SPARQL query over the artifacts dataset retrieving resources having the same typology as the selected one. Listing 1 depicts an example of sparql query going through the knowledge graph structure where the typology can easily retrieved since be the 'hasTypologyName' and 'hasName' are directly related data properties to the vertex (concept) 'Amphorae'.

Listing 1: A SPARQL query to retrieve artifacts having similar typology as 'Amphore_A02'

```
PREFIX arp:<http://www.arpenteur.org/ontology/Arpenteur.owl#>
```

```
SELECT distinct ?similarName
```

WHERE {

```
?amp a arp:Amphorae;
arp:hasName "Amphore_A02";
arp:hasTypologyName ?typo.
```

```
?similar a arp:Amphorae;
arp:hasTypologyName ?typo;
arp:hasName ?similarName.
```

```
Filter (?amphorae != ?amp)
}
```

<footnote n="14">¹⁴https://datahub.ckan.io/dataset/xlendiamphorae</footnote>

<footnote n="15">¹⁵http://www.arpenteur.org/ontology/temporal/20180921.html</footnote>
* **Select similar artifacts having a Hausdorff distance less than a given threshold:** A SPARQL query over the artifacts dataset to retrieve the shape (the theoretical 3D models) representing each one of the compared models. Listing 2 shows an example of a SPARQL query for retrieving the 3D shapes, i.e. a set of theoretical 3D points retrieved via the directed edge (object property) 'hasMeasuredPointManager' that relates to the concept 'Amphorae' in the ontology. The two shapes are aligned into a common coordinate reference system by retrieving the corresponding orientation matrices and translations. The hausdorff measure is performed to compare the two 3D shapes.

<caption>Listing 2: A SPARQL query to retrieve 'Amphore_A02' 3D shape</caption>

```sparql
PREFIX arp:<http://www.arpenteur.org/
ontology/Arpenteur.owl#>

SELECT distinct ?x ?y ?z
WHERE {
    ?amp a arp:Amphorae;
    arp:hasName "Amphore_A02";
    arp:hasMeasuredPointManager ?pts.
    ?pts arp:hasASetOf3DPointWithObs ?pt.
    ?pt arp:hasX ?x;
    arp:hasY ?y;
    arp:hasZ ?z.
}
```

* **Select similar artifacts having a given dimension property (width, height, length or volume) value difference less than a given threshold:** A SPARQL query over Xlendi artifact dataset to crawl the dimension property values and select the similar ones based on the threshold. As can be seen in Listing 3, the height value corresponding to an instance of the concept 'Amphorae' can be retrieved via object property 'hasHeight' which is inherited (inferred) from the concept 'SpatialObject' in the Arpenteur ontology.

<caption>Listing 3: A SPARQL query to retrieve artifacts having height distance less than 5 cm from 'Amphore_A02'</caption>

```sparql
PREFIX arp:<http://www.arpenteur.org/
ontology/Arpenteur.owl#>

SELECT distinct ?similarName ?distance
WHERE {
    ?amp a ?type;
    arp:hasName "Amphore_A02";
    arp:hasHeight ?h1.
    ?similar a ?type;
    arp:hasName ?similarName;
    arp:hasHeight ?h2.
    BIND(ABS(?h1 - ?h2) AS ?distance).
    Filter(?distance < 0.05).
    Filter(?type IN
            (arp:Amphorae, arp:GrindingStone)).
    Filter (?similar != ?amp).
}
```

* **Get the survey images for the source artifact for a given excavation:** While artifacts desciption are stored in the artifacts dataset, the photographs information are stored within different temporal datasets corresponding to survey dates. Hence, for a selected source artifact, a federated SPARQL query is performed over:

- the artifacts dataset to retrieve the bounding box corresponding to the selected source.

- the temporal dataset to select the set of 3D points that fits into the bounding box and return the corresponding photographs.

Listing 4 shows an example of a query performed directly over the SPARQL endpoint (HTTP query interface) of the temporal dataset while an embedded SPARQL service will allow queering the Xlendi artifact dataset in the body of the same query. For a given 3D point, the corresponding photograph can be retrieved via the ontology properties path: 'hasImagePointManager/hasASetOfObservation/hasPhotograph/hasFullFileName'.

<caption>Listing 4: A federated SPARQL query to retrieve photograph corresponding to 'Amphore_A02'</caption>

```sparql
PREFIX arp: <http://www.arpenteur.org/
ontology/Arpenteur.owl#>
SELECT distinct ?photograph WHERE {
    ?pt3D arp:hasX ?x;
    arp:hasY ?y;
    arp:hasZ ?z;
    arp:hasImagePointManager
    /arp:hasASetOfObservation
    /arp:hasPhotograph
    /arp:hasFullFileName ?photograph.
}
SERVICE <Xlendi Artifact endpoint URL>
{
    SELECT distinct * WHERE {
        ?amp arp:hasName "Amphore_A02";
        arp:hasBoundingBox ?bb.
        ?bb arp:hasXMax ?xMax;
        arp:hasXMin ?xMin;
        arp:hasYMax ?yMax;
        arp:hasYMin ?yMin;
        arp:hasZMax ?zMax;
        arp:hasZMin ?zMin.
    }
    Filter (?x < ?xMax && ?x > ?xMin && ?y < ?yMax && ?y > ?yMin && ?z < ?zMax && ?z > ?zMin)
}
```

## 4.2 2D/3D Web-based Tools

Our Web tools offer the user a 2D and 3D interactive views of Xlendi shipwreck. The 2D and 3D Web tool are depicted respectively in figure 4 and Figure 5.

Most often a 2D representation in plan is better than a 3D representation. In addition, the lack of standard to express 3D models
<figure><img src="image_9.png" /><figcaption>Figure 5: 3D view of *Xlendi* Orthophoto in our 3D web tool (powered by 3DHop).</figcaption></figure>

makes portability difficult. Nevertheless the 2D representations must be generated from 3D models to ensure the necessary accuracy. In this work we have opted for the two modes of representation: 2D using the orthophotos themselves generated from the 3D models resulting from photogrammetric surveys. Figure 4 is obtained from an orthophoto produced with the photogrammetric survey obtained in 2014. We used the Raphaël JavaScript Library<fnref n="17" /> to represent the 2D implementation of different artifacts in *Xlendi* site.

We have chosen not to directly use the orthophoto as an interface to the end user because, despite the metric qualities of such a document, a key aspect of this work is missing: the human interpretation of the knowledge-based site.

The drawing produced here is by Mrs. Gina De Angelis, architect, who interpreted the orthophoto and produced this drawing by identifying each artifact. The two blue squares are the quadrats used for the excavation in 2018, the colors code the year of removal of the artifacts.

The 3D Web tool is accessible via the URL<fnref n="18" />. We provide a 3D view of the different models corresponding to *Xlendi* artifacts. The artifact positions map the 3D views of the *Xlendi* orthophotos corresponding to a given site survey period. The 3D viewer is powered by 3DHop (Potenziani et al., 2015). As described in Section 4.1, an advanced information analysis of the *Xlendi* site is presented to the user via different user-friendly queries (SPARQL queries are performed in back-end of the Web tool). Moreover, the web tools contain download options for 3D files corresponding to *Xlendi* artifacts and their typologies.

We draw the reader intention that our main intuition behind our Web tools is to provide a panoramic bird's-eye view for *Xlendi* shipwreck to allow a maximum exploitation of the available CH information by users (typically archaeologists). Let us take the

<figure><img src="image_10.png" /><figcaption>Figure 6: A Sparql query result tab view of the morphological attributes corresponding to Amphorae A02 as the selection in the Figure 4</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure 7: An image display tab of artifacts in the *Xlendi* 2D Web tool corresponding to the Amphorae A02 as the selection in the Figure 4</figcaption></figure>

example of amphorae A02, a panoramic view of this artifacts consists of:

* a drawing view of the *Xlendi* orthophoto where the amphorae A02 is colored in red, see Figure 4.

* the set of morphological attributes describing this amphorae can be fetched via the SPARQL tab in our 2D web tool, as shown in Figure 6.

* a 2D image of Amphorae A02 can be depicted via the tab Image in the 2D Web tool, as can be seen in Figure 7. In the 3D Web tool, this image is displayed automatically on artifacts click.

* a 3D model of Amphorae A02 can be displayed in our 2D Web tool via the tab Model_3D, as depicted in Figure 8. The 3D model is also available for downloaded via a user-friendly query.

* a 3D view of the complete *Xlendi* site highlighting in red

<footnote n="17">¹⁷https://dmitrybaranovskiy.github.io/raphael</footnote>

<footnote n="18">¹⁸http://www.lsis.org/groplan/hop/xlendiTimeLine/xlendi.html</footnote>
<figure><img src="image_12.png" /><figcaption>Figure 8: A 3D view display tab (powered by 3DHop) of artifacts in the *Xlendi* 2D Web tool corresponding to the Amphorae A02 as the selection in the Figure 4.</figcaption></figure>

the position of the amphorae A02 shape within the shipwreck, as can be seen in Figure 5.

* available photographs taken in the UCH site of the selected amphorae A02 and an orthophoto depicting the complete shipwreck where the amphorae A02 is colored in yellow, see Figure 9. Note here that different surveying photos and orthophotos are available for different survey dates and can be fetched via a user-friendly query in our 2D/3D Web tools.

<figure><img src="image_13.png" /><figcaption>Figure 9: *Xlendi* orthophoto and photographs corresponding to the selected amphorae A02.</figcaption></figure>

## 5. CONCLUSION

This paper presents an approach for retrieving, via the Web, photogrammetric data as well as non graphical data without any computation but only with querying on several semantic datasets acquired over the time. This is performed following Semantic Web best practices, where *Xlendi* data is modeled by the Arpenteur ontology and stored in an online published linked

open dataset providing an online SPARQL service. The web approach presented here offers request on surveys data made on *Xlendi* wreck from 2009 to 2018.

This project addresses two types of problems: first the link between representation and the site. We see it in the 2D interface that is proposed, the statement while relying on a precise 3D measurement technique here is a 2D interpretation of the photogrammetric model. The interpretation is performed by a human, a specialist in the field who relies on incomplete metrics and his knowledge of the field. This 2D document, produced by an expert, highlights the geometry of the artifacts studied and represents only that one. In the 3D interface also proposed in this project, both the web interface and the Virtual Reality interface, the 3D model is there as an interface to non-graphical data and also supports the expression of useful model visible parts, among others, to evaluate the ratio measure/deduction. Hence, in this project, we chose a knowledge graph solution in which the semantic data are stored in a database of graphs and represented by a set of aligned ontologies. This approach allows us to turn resolutely towards an approach that allows sharing, exchanging, reusing and analysis of archaeological data in an effort to standardize formats.

Parallel to the website with a 2D/3D interface we are currently working on a virtual reality interface. The operator in total immersion in the 3D model can, on the one hand, follow the evolution of the morphology of the site over the years, and on the other hand visually assess the relevance of the choice of the morphology of the artifacts. The combined use of photogrammetry process, 3D measurements, 3D/2D visualization tools and an ontological model provides the ability to analyze and compare data in a way that we have not yet finished exploring.

## ACKNOWLEDGEMENTS

The authors would like to thank the i-MareCulture project for partially funding this work, URL: https://imareculture.eu/.

## REFERENCES

Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z., 2007. Dbpedia: A nucleus for a web of open data. *The semantic web*, Springer, 722–735.

Auer, S., Lehmann, J., Ngomo, A.C.N., Zaveri, A., 2013. Introduction to linked data and its lifecycle on the web. *Reasoning Web. Semantic Technologies for Intelligent Data Access*, 1–90.

Ben Ellefi, M., Nawaf, M.M., Sourisseau, J.C., Gambin, T., Castro, F., Drap, P., 2018a. Clustering over the cultural heritage linked open dataset: Xlendi shipwreck. *SW4CH@ESWC*.

Ben Ellefi, M., Papini, O., Merad, D., Boi, J.M., Royer, J.P., Pasquet, J., Sourisseau, J.C., Castro, F., Nawaf, M.M., Drap, P., 2018b. Cultural heritage resources profiling: Ontology-based approach. *Companion of the The Web Conference 2018*, WWW ’18, 1489–1496.

Doerr, M., 2009. Ontologies for cultural heritage. *Handbook on Ontologies*, Springer, 463–486.

Drap, P., Merad, D., Hijazi, B., Gaoua, L., Nawaf, M.M., Saccone, M., Chemisky, B., Seinturier, J., Sourisseau, J.C.,
Gambin, T., Castro, F., 2015. Underwater Photogrammetry and Object Modeling: A Case Study of Xlendi Wreck in Malta. *Sensors*, 15, 30351–30384. http://www.mdpi.com/1424-8220/15/12/29802.

Gaitanou, P., Gergatsoulis, M., Spanoudakis, D., Bountouri, L., Papatheodorou, C., 2016. *Mapping the Hierarchy of EAD to VRA Core4.0 Through CIDOC CRM*. Springer International Publishing, Cham, 193–204.

Gambin, T., Chemisky, B., Drap, P., 2018. Exploring the Phoenician shipwreck off Xlendi bay, Gozo. A report on methodologies used for the study of a deep-water site. *Underwater Technology The International Journal of the Society for Underwater*, 35, 71-86.

Guarino, N., Oberle, D., Staab, S., 2009. What is an ontology? *Handbook on ontologies*, Springer, 1–17.

Lóscio, B.F., Burle, C., Calegari, N., 2017. Data on the web best practices. Technical report, W3C. https://www.w3.org/TR/2017/REC-dwbp-20170131.

Niccolucci, F., 2017. Documenting Archaeological Science with CIDOC CRM. *Int. J. Digit. Libr.*, 18, 223–231. https://doi.org/10.1007/s00799-016-0199-x.

Niccolucci, F., Hermon, S., 2017. Expressing Reliability with CIDOC CRM. *Int. J. Digit. Libr.*, 18, 281–287. https://doi.org/10.1007/s00799-016-0195-1.

Potenziani, M., Callieri, M., Dellepiane, M., Corsini, M., Ponchio, F., Scopigno, R., 2015. 3DHOP: 3D Heritage Online Presenter. *Computers Graphics*, 52, 129 - 141.

Rietveld, L., Hoekstra, R., 2017. The YASGUI family of SPARQL clients 1. *Semantic Web*, 8, 373–383.

Scaradozzi, D., Sorbi, L., Zoppini, F., Scaradozzi, D., Gambogi, P., 2013. Tools and techniques for underwater archaeological sites documentation. *2013 OCEANS - San Diego*, 1–6.

Scaradozzi, D., Zingaretti, S., Panebianco, L., Altepe, C., Egi, S., Palma, M., Pantaleo, U., Ferraris, D., Micheli, F., 2017. Docuscooter: a novel robotics platform for marine citizen science.