# Computation of fuzzy joins over large collections of JSON data using semantic similarity

Alan Petit, Matthew Damigos, Laurent D Orazio, Eleftherios Kalogeros

<figure><img src="image_1.png" /></figure>

## To cite this version:

Alan Petit, Matthew Damigos, Laurent D Orazio, Eleftherios Kalogeros. Computation of fuzzy joins over large collections of JSON data using semantic similarity. 2026. hal-05505051

HAL Id: hal-05505051

https://hal.science/hal-05505051v1

Preprint submitted on 11 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Computation of fuzzy joins over large collections of JSON data using semantic similarity

Alan Petit<fnref n="1" />*, Matthew Damigos<fnref n="2" />, Laurent D'Orazio<fnref n="3" /> and Eleftherios Kalogeros<fnref n="2" />

**Abstract**

In recent years, fuzzy joins have emerged as a critical tool in data integration, data cleaning, and similarity-based retrieval tasks, especially when dealing with noisy or heterogeneous data. Although fuzzy joins have been widely explored for relational data, existing approaches typically focus on textual similarity, matching records based on certain text values and text distances. Apart from relational data, syntactic similarity of XML and JSON documents has also been investigated, where the matching documents are compared over their hierarchical structure. These methods often struggle when confronted with variations in element names, formats, or hierarchical layouts. In contrast, the semantic similarity of semi-structured data, such as JSON documents, remains underexplored, despite its importance in capturing the true meaning conveyed by document values. This paper addresses the problem of performing fuzzy joins between JSON documents by comparing their semantic content rather than their structural form. We propose and describe several methods based on document embeddings, aiming to capture the meaning of a document from its values and contextual relationships. The proposed techniques offer a promising foundation for semantic joining tasks in semi-structured data environments.

**Keywords**

Fuzzy join, JSON similarity, semantic similarity, embeddings, structural comparison, distributed computation

## 1. Introduction

The growing volume and heterogeneity of semi-structured data, particularly JSON (JavaScript Object Notation) documents, poses significant challenges for data integration and analysis. In particular, organizations need more and more to integrate, deduplicate, and correlate JSON documents that describe the same real-world entities or events but originate from different sources and production pipelines. Such documents are rarely uniform. They may use different key names (e.g., author vs. creator), different value conventions and different nesting schemes, resulting a variety of differences in both structure and content. As a result, conventional equality joins, which rely on strict equality between attributes, fail to identify such correspondences. The problem becomes more challenging when the volume of the input datasets is large.

Fuzzy joins address this limitation by replacing strict equality with a similarity measure; i.e., returning pairs of documents whose similarity exceeds a predefined threshold [1]. While fuzzy-join techniques have been extensively studied for relational data and for text-centric matching (e.g., edit distance, Jaccard, cosine over token sets) [1, 2, 3, 4, 5, 6, 7], to evaluate fuzzy joins over collections of semi-structured data is not straightforward since not only the text-values but also the hierarchical structure need to be taken into account. Since JSON data is self-descriptive, multiple structures of the same information might be found. To overcome this issue, computing structural similarity has been extensively investigated in the past for tree structures (e.g., [33, 34, 35]), XML documents (e.g., [29, 30]), and graphs (e.g., [31, 32]). In the last decade, structural similarity over JSON document has also attracted attention [36, 37, 38, 39],

DOLAP'26: International Workshop on Design, Optimization, Languages and Analytical Processing of Big Data, March 24, 2026,
Tampere, Finland

* Corresponding author.

<figure><img src="image_3.png" /></figure>

alan.petit04@gmail.com (A. Petit); mgdamigos@ionio.gr (M. Damigos); laurent.dorazio@univ-rennes.fr (L. D'Orazio);

kalogero@ionio.gr (E. Kalogeros)

<figure><img src="image_4.png" /></figure>

0000-0003-0431-1482 (M. Damigos); 0000-0001-8614-1848 (L. D'Orazio); 0000-0001-8080-6343 (E. Kalogeros)

<figure><img src="image_5.png" /></figure>

© 2026 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

<footnote n="1">¹Polytech Nantes, Nantes Université, Nantes, France</footnote>

<footnote n="2">²Department of Archives, Library Science & Museology, Ionian University, Corfu, Greece</footnote>

<footnote n="3">³Univ Rennes, CNRS, IRISA, Lannion, France</footnote>
where the main challenge is to support both ordered (i.e., arrays) and unordered (i.e., objects) set of
values. These approaches [36, 37, 38, 39] focus on variations of tree edit distance to measure structural
similarity, and focus on single-machine computation. Taking into account both structure and content
of JSON documents, however, is still an open problem.

In this work, we present a distributed evaluation method for computing fuzzy joins over large
collections of JSON data, handling similarity over both content and structure. The proposed approach
uses methods for converting JSON documents first to text and then to document embeddings, aiming to
preserve both content and structure through descriptive representations of the information encoded in
the JSON documents. In particular, the main contribution of this paper are summarized as follows:

* We propose a novel approach for evaluating fuzzy joins over collections of JSON documents, by using a two-step transformation of each document to a document embedding that captures the meaning of the information encoded in the JSON documents.

* We present five different transformation approaches of a document to its text-representation, where the four of them use specialized prompts posed on an LLM.

* We present scalable MapReduce algorithms for computing the fuzzy joins over large collections of generated embeddings, avoiding shuffling the JSON documents during join computation.

The remaining part of this paper is organized as follows. Section 2 presents the preliminary definitions and concepts used throughout this paper. Section 3 provides an overview of the proposed approach, while Sections 4 and 5 present the JSON-to-text transformation and the distributed computation methods. Section 6 presents a preliminary experimental analysis, and Section 7 analyzes the related work. Finally, Section 8 concludes and discusses future work.

## 2. Preliminary Definitions

In this section, we discuss the main concepts used throughout this paper, and initially, we focus on JSON (JavaScript Object Notation) [10] documents. A JSON document is either a possibly empty object (unordered list enclosed into curly brackets `{ }`) of key-value pairs or a possibly empty array of values (ordered list enclosed into square brackets `[ ]`), where (a) the values could be either objects or arrays, or primitive values (i.e., string, number, boolean, or null), and (b) the keys are strings. The keys are unique within an object. A *JSON collection* (or simply, a *collection*) is a multiset/bag of JSON documents, which do not necessarily share the same structure.

To navigate into and query JSON documents, we consider JSONPath patterns [11, 12]. JSONPath is a query language that is inspired by XPath and is mainly used to navigate into a single JSON document. JSONPath patterns are recursively constructed using the navigation expressions, such as the symbol `$` which represents the root element, the child operator (`.` or `[]`), and the descendant operator (`..`). Note that in an array, the position of each value could be considered as the key of the value at this position. The output of a JSONPath expression *P* when it is applied on a JSON document *J* is given by a JSON document and is denoted as *P(J)*.

### 2.1. Distributed Computation

MapReduce [13] is a parallel and distributed programming model to process large amounts of data on clusters of commodity machines. This model allows users to focus on designing their applications regardless of the distributed aspects of the execution. Figure 1 illustrates the MapReduce execution.

A MapReduce program consists of two distinct phases, namely, the Map phase and the Reduce phase. Each phase performs a user-defined function on a key-value pair. The function Map (M) takes a pair of entries (k1, v1) and emits a list of intermediate pairs (k2, v2): (k1, v1) $\xrightarrow{map}$ (k2, v2). The intermediate values associated with the same key k2 are grouped together and then transmitted to the
<figure><img src="image_6.png" /><figcaption>Figure 1: MapReduce Execution</figcaption></figure>

Reduce function, which applies the predefined computation over the values of a certain group (e.g., aggregation operator): $(k2, [v2, ...]) \xrightarrow{reduce} (k3, v3)$.

A MapReduce program is executed on multiple compute nodes. During the Map phase, each Map task reads a subset (called split) of an input dataset and applies the Map function for each key/value pair. The system supports grouping of intermediate data and sends them to the relevant nodes to apply the Reduce phase. This communication process is called Shuffle. Each Reduce task collects the key/value pairs of all the Map tasks, sorts/merges the data with the same key and applies the Reduce function to generate the final results.

## 2.2. Large Language Models (LLMs) and Embeddings

Large Language Models (LLMs) [14] are sophisticated neural architectures developed using huge collections of text data to forecast and produce coherent, contextually appropriate natural language. They generally use transformer architectures [15], where self-attention mechanisms allow the model to identify long-range dependencies in text. Notable instances include GPT (Generative Pre-trained Transformer) [16] and its subsequent versions (e.g., LlaMA [17]), which vary in parameter quantity, training data, and tuning approaches. LLMs are used in a wide variety of linguistic and reasoning tasks, including summarization, question answering, translation, and verbalization of structured data, which makes them ideal for creating narratives about museum objects from formal ontological data.

Effective utilization of LLMs requires prompt engineering [18], a process of designing LLM inputs (a.k.a., prompts - e.g., questions, instructions, context, and examples) that guide the model's behavior toward a desired output (specific content, tone, or structure). In this context, a large number of prompting methodologies have been proposed, and they keep evolving rapidly, introducing sophisticated strategies to enhance controllability and task alignment of LLM-generated outputs. These methodologies range from simple instruction-based expressions (e.g., zero-shot prompting [19]) to multi-stage, reasoning-oriented (e.g., chain-of-thought prompting [20]), and retrieval-augmented generation (RAG) [21] approaches, reflecting the growing need to balance creativity with consistency and domain particularity.

Apart from text generation, LLMs rely on and produce dense vector representations of linguistic elements, called *embeddings*. An embedding maps distinct elements (e.g., words, sentences, or documents) into points within a continuous, low-dimensional vector space so that geometric relationships (e.g., cosine similarity, Euclidean distance) represent semantic similarity [22]. Early work on embeddings focused on *word embeddings* [23], i.e., vector representations for individual word types.

Embeddings are learned from large text corpora, and provide a reusable interface of the raw text. Transformer-based LLMs produce contextual token (i.e., a single segment of text that the model perceives as a fundamental symbol) embeddings, where each token's representation depends on the entire surrounding sequence through self-attention [24]. For many applications, however, it is desirable to represent entire sentences, paragraphs, or short documents as single vectors. *Sentence embeddings*
(or, document embeddings) [25, 26] address this need by mapping variable-length text spans into fixed-dimensional vectors that are directly comparable via standard similarity measures.

## 2.3. Fuzzy joins

A fuzzy join aims to combine data based on their similarity (i.e., how similar are two elements). The similarity between two elements is typically expressed as a continuous score ranging from 0 to 1, where 1 indicates perfect equivalence and 0 denotes complete dissimilarity. The corresponding distance can be defined as $1 - \text{similarity}$.

Text similarity is the most common case and is widely-used. It relies on a distance measure to find all pairs $(x, y)$ in the input datasets with a distance $(d)$ below some pre-defined threshold $\tau$ [27, 7], where the distance function could be one of various metrics [28], such as Edit Distance (Levenshtein Distance), Jaro-Winkler, Jaccard distance, Cosine distance. Formally, given a threshold parameter $\tau \in [0, 1]$, the fuzzy join between two datasets $C_1, C_2$ is defined as follows, where $d(J_1, J_2)$ is a distance function applied over the elements $J_1 \in C_1, J_2 \in C_2$:

$$C_1 \bowtie_{d \le \tau} C_2 = \{(J_1, J_2) | J_1 \in C_1, J_2 \in C_2, d(J_1, J_2) \le \tau\}.$$

In this definition, the datasets could be collections of strings/texts, JSON or XML documents. It also straightforwardly extended to apply to relational datasets, where the $C_1, C_2$ are relations and $J_1, J_2$ are values of specific attributes or entire tuples. Furthermore, a distance function could be replaced by a similarity function $sim$, following their inverse relation: $d(J_1, J_2) = 1 - sim(J_1, J_2)$.

In this paper, we use Cosine similarity $sim_{cos}$ (or, the corresponding distance $d_{cos} = 1 - sim_{cos}$) of two vector representations $\vec{V}_1, \vec{V}_2$ of the collection elements $J_1, J_2$, respectively, to evaluate their similarity, which is defined as follows: $sim_{cos}(\vec{V}_1, \vec{V}_2) = \frac{\vec{V}_1 \cdot \vec{V}_2}{||\vec{V}_1|| \cdot ||\vec{V}_2||}$, where $\vec{A} \cdot \vec{B}$ is the dot product of the two vectors, and $||\vec{A}||$ and $||\vec{B}||$ are the Euclidean norms (L2 norms) of the vectors.

To join collections of complex structures, such as nested, tree-structured data (e.g., JSON or XML documents) and graphs, their structural information could significantly affect the similarity of collection elements. Unlike flat records where comparison is often attribute-wise, comparing complex structures requires specialized distance functions that take into consideration elements' structure. Different types of distances (a.k.a. structural similarity distances - e.g., Tree Edit Distance, Path Similarity) are employed in order to capture differences in the structure (e.g., [29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39]). In this work, we focus on collections of JSON documents and aim to evaluate their similarity, taking into account both content and structure.

# 3. Fuzzy join based on semantic - An overview similarity

This section focuses on presenting the main methodology followed to compute a fuzzy join of two collections of JSON documents, named FLESHJ-Join (stands for Fuzzy LLM-based Embedding Similarity for Heterogeneous JSON), using semantic similarity. Note that the collections do not necessarily conform to a specific JSON schema. In particular, the JSON documents in each collection may exhibit significant heterogeneity in their structure (nesting, key names, ordering of fields) and primitive values. Traditional fuzzy joining approaches that rely on either textual similarity of certain values or structural similarity of the JSON documents are insufficient in this context, as they fail to capture the underlying semantic meaning of the data. For example, the JSON documents $J_1$ and $J_2$ illustrated in Figure 2 describe the same real-world entities using entirely different structures and vocabularies, yet are semantically highly similar.

To address this limitation, our approach decouples the representation of JSON documents from the join operation itself. The core idea is to transform each JSON document of each collection into a dense vector representation (document or sentence embedding) that encodes its semantic content, and then to join collections by comparing these embeddings using cosine similarity. Figure 3 illustrates an overview
<figure><img src="image_7.png" /><figcaption>Figure 2: Example - Semantically similar and dissimilar JSON documents</figcaption></figure>

of the proposed semantic fuzzy-join framework, which is organised into an offline phase and an online phase.

In the offline phase, two main processing steps as considered: (1) generating a text-representation that preserves the JSON semantics, and (2) encoding this representation into a fixed-dimensional vector space where semantic proximity corresponds to geometric closeness. In the former step, each JSON document in the source collections $C_1, C_2$ is processed and transformed (e.g., by leveraging an LLM) into a natural-language representation. This step produces two collections $\mathcal{T}_1$ and $\mathcal{T}_2$, respectively, of text-representations, a text-representations for each JSON document. In addition, we keep a mapping of the original JSON documents to its text-representations. The text-representations are then passed to an embedding generation component, which uses a document or sentence embedding model to compute vector representations for each description, yielding two embedding collections, $\mathcal{E}_1$ and $\mathcal{E}_2$. These embeddings are stored in permanent storage and reused during query execution. In essence, the collections of embeddings are used as indexes during join evaluation.

<figure><img src="image_8.png" /><figcaption>Figure 3: Overall processing pipeline of FLESHJ-Join</figcaption></figure>

In the online phase, the precomputed embeddings $\mathcal{E}_1$ and $\mathcal{E}_2$ are fed into a fuzzy-join execution module, which computes semantic similarity scores (e.g., cosine similarity) between vectors and identifies pairs whose similarity exceeds a given threshold. The resulting pairs of embeddings are then translated back to the corresponding source JSON documents in $C_1$ and $C_2$, so that the final join result is expressed directly in terms of the original JSON data. The separation of these two phases ensures scalability, reusability and efficiency of the join computation, since time-consuming LLM and embedding computations are performed offline, while keeping the online fuzzy-join step efficient and scalable.

Formally, the problem tackled is defined as follows. Consider two collections of JSON documents $C_1$ and $C_2$, so that each document $J_i \in C_1 \cup C_2$ is associated with a unique identifier $id(J_i)$. Suppose two transformations $t_1, t_2$ transforming a JSON document to one or more texts, and a text to a fixed-length numerical vector (document embedding), respectively. Let $\mathcal{T}_1, \mathcal{T}_2$ be the collections produced by applying $t_1$ on JSON documents of $C_1, C_2$, and $\mathcal{E}_1, \mathcal{E}_2$ be the collections produced by applying $t_2$ on texts of $\mathcal{T}_1, \mathcal{T}_2$, respectively, so that the unique identifiers of the JSON documents in $C_1, C_2$ are retained in $\mathcal{T}_1, \mathcal{T}_2$ and $\mathcal{E}_1, \mathcal{E}_2$. The FLESHJ-Join is defined as follows, where $\tau$ is a given similarity threshold:

$$C_1 \bowtie_{\ge \tau} C_2 = \{[J_1, J_2] \mid J_1 \in C_1, J_2 \in C_2, \text{sim}_{\text{cos}}(t_2(t_1(J_1)), t_2(t_1(J_2))) \ge \tau\}.$$
Considering, now, two large collections $C_1, C_2$ of JSON documents, we want to compute, efficiently, the $C_1 \bowtie_{\ge \tau} C_2$ by maximizing the semantic accuracy. In particular, taking into account the collection $\mathcal{O}$ of pairs of JSON documents that are semantically similar, i.e., the collection representing the desired output, for a given similarity threshold, we want $(C_1 \bowtie_{\ge \tau} C_2) - \mathcal{O} \simeq \emptyset$, while minimizing both false positives (spurious matches) and false negatives (missed matches). Typically, semantic accuracy can be assessed in terms of standard information retrieval metrics such as precision, recall and F1 score, defined over a gold-standard set of matching and non-matching pairs. High semantic precision indicates that most pairs returned by FLESHJ-Join indeed correspond to semantically similar JSON documents, while high semantic recall indicates that most of the true semantically similar documents in $C_1 \times C_2$ are retrieved by the join.

# 4. Transforming JSON documents into text representations and embeddings

The primary challenge lies in designing scalable and effective methods for representing JSON documents through natural-language text. In this context, the first step consists in transforming each JSON document $J \in C_1 \cup C_2$ into a natural-language text-representation $t_1(J) \in \mathcal{T}$, using a transformation function $t_1: C_1 \cup C_2 \to \mathcal{T}_1 \cup \mathcal{T}_2$. The goal of this transformation is twofold: (i) to preserve as much as possible of the semantic content of the original document (i.e., what real-world entity or concept it describes), and (ii) to encode enough information about the document's structure (i.e., how values are related to each other and what the mean) so that produced document embeddings can capture similarities that depend on both content and structure. At the same time, the text-representations must be compact and homogeneous in style, in order to avoid unnecessarily long inputs and to keep the behavior of the embedding model stable.

To generate the representative text from a certain JSON document, we consider two main methods. The first method simply generates the text by following a certain text-generation rules. The second one uses an LLM to generate this text-representation. In both cases, such a transformation is applied independently to each document, which makes it suitable for large-scale, parallel processing frameworks (e.g., Spark, or MapReduce). In each case, the resulting texts form the collections $\mathcal{T}_1$ and $\mathcal{T}_2$, which are subsequently passed to the embedding model (transformation $t_2$) to obtain the vector collections $\mathcal{E}_1$ and $\mathcal{E}_2$ used during the online fuzzy-join phase. In the following sections, we focus on generating the text-representations from a JSON document. As for the transformation of the generated text-representation to an embedding, we simply consider a sentence/document embedding model (i.e., Sentence-transformer models or a small LLM), and ensure that the same model is used for each text-representation.

## 4.1. Text-representation using text-generation rules

Focusing on the first approach, we define a set of text-generation rules that reflect both the content and the structure of the data. Initially, we model each JSON document as a rooted tree whose nodes are either keys, primitive values, or array item identifiers, where the root node represents the whole document. Formally, given a JSON document $J$, we construct a rooted, labeled tree $T(J) = (V, E, r, \ell)$ as follows. The root node $r \in V$ is a special *document node* corresponding to the entire JSON document. Every other node $v \in V - \{r\}$ belongs to exactly one of the following types:

* **Key nodes:** For each JSON object and for each field of the object with key $k$, we create a node labelled with $k$. An edge is added from the node representing the enclosing object (or the document root) to this key node. Key nodes represent the logical attributes of the document.

* **Array item nodes:** For each JSON array of objects and for each element at position $i$ in this array (with $i \in \{0, \dots, n-1\}$), we create an item node labeled as "Item $i$". An edge is added from the node representing the array (typically a key node) to this item node.
* **Value nodes:** For each primitive value *v* (string, number, boolean, or null), either it is included in an array or it is the value of certain key, we create a leaf node labeled with that value. An edge is added from the node representing its enclosing key or item to the corresponding value node.

To construct such a tree, we follow a top-down approach, starting from the document node. Then, we create its child nodes from either its key-attributes (if the root element is an object) or its item nodes (if the root element is an array of object), or its value nodes (if the root element is an array of primitive values). Next, we recursively follow the approach described previously to construct the tree-representation of the JSON document. Note here that we ignore encoding the position/index of each value in an array of primitive values since we do not want to have an impact of the position of the values on the similarity. In addition, we encode the nested arrays of primitive values using their flattened form; e.g., the array ["a", "b"], "c"] is similarly treated as the array ["a", "b", "c"). For example, in Figure 2, the tree-representation of document *J*₁ is the tree *T*(*J*₁).

Based on the tree representation *T*(*J*) of a JSON document *J*, we define a set of text-generation rules that map nodes and their children to short, declarative sentences. In particular, starting from the root, for each node *n* ∈ *T*(*J*) having non-leaf values, we generate a sentence of the form: "[n] has fields *k*₁, ..., *k*ₚ", where *k*₁, ..., *k*ₚ are the children of *n* ∈ *T*(*J*). Especially in the case of root node, the sentence is modified to "document is described by *k*₁, ..., *k*ₚ". For each node *n* ∈ *T*(*J*) having at least one leaf value, we generate a sentence of the form: "[n] has values: *x*₁, ..., *x*ₙ". If *n* has a single leaf node "v", then the sentence can be modified to "The value of [n] is [v]". For example, for the tree-representation *T*(*J*₁) of *J*₁, we end up with the following text-representation: "document is described by actors. actors has fields Item1, Item2. Item1 has fields name, role. The value of name is Matthew. The value of role is scientist. Item1 has fields name, role. The value of name is Julie. The value of role is artist". Note here that since multiple nodes might have the same key, we could use the reachable path of each node from the root, instead of its single label.

## 4.2. Text-representation using LLMs

In this section, we focus on generating a text-representation of a JSON document by using LLMs; i.e., the text-generation transformation is defined by prompt posed on an LLM. To do so, we investigate several prompts aiming to capture both the structure and content. We mainly focus on Zero-Shot Prompt design, including instructions, constraints and roles. Rather than relying on keys, values, or structural patterns, the goal is to produce a text-representation that captures the true meaning of the document.

In this context, we end up with four different prompts, which are depicted in Figure 4. Each prompt is designed to ask the LLM to behave as a JSON-to-text converter/transformer (role-based prompting). Additionally, it specifies a set of instructions that are important for FLESHJ-Join, focusing on faithfulness to the input (no hallucinated attributes or values) and a concise, fluent output in a standardised format, so that the important keys and values, and the hierarchical relationships are covered. The prompts proposed can be combined or selected depending on the desired trade-off between faithfulness, structural emphasis, and readability.

The first prompt (see Prompt 1 in Figure 4) includes a short set of instructions, asking the model to generate a natural-language description for a given a single JSON document that includes all important keys and values while reflecting their hierarchical relationships, without inventing information. The output is constrained to a single paragraph of plain text, and the JSON is not repeated in the answer. This prompt explicitly focus on preserving both content (values) and structure (groupings and nesting), and producing compact descriptions.

The second prompt (see Prompt 2 in Figure 4) uses a high-level template, in which the model is asked to summarize the meaning of the JSON document by capturing: (i) the type of information (i.e., describing the high-level nature of the content), (ii) the main entities or concepts covered in the document, and (iii) the key relationships or patterns relate these entities (i.e., describing how these entities are related or semantically organized). The prompt explicitly asks the model to avoid returning
<figure><img src="image_9.png" /><figcaption>Figure 4: Prompts transforming a JSON document to a text-representation.</figcaption></figure>

technical information related to the structure or format unless it provides semantic information. This representation focuses on describing the role of the document in a broader data integration scenario (e.g., what it is about and which entities it ties together), which can help embeddings align documents that fulfill the same function but differ in structure.

The third prompt (see Prompt 3 in Figure 4) asks the model to produce a short paragraph capturing the information included in the JSON document; similar to Prompt 2. Prompt 3, however, focuses on the information that is explicitly present in the JSON, trying to avoid inferences, interpretations and opinions. In this context, it asks for certain strict rules to be applied, such as no interpretations or generalizations, no inferred relationships beyond what is explicitly indicated, no background knowledge. This variant prioritizes faithfulness over readability and abstraction, making it suitable when the main objective is to minimize any risk of semantic inconsistencies between the JSON document and its text-representation.

The fourth prompt (see Prompt 4 in Figure 4) defines a zero-shot, role-based, process-guided instruction asks the LLM to convert a JSON document into a short, human-readable description using a limited number of simple sentences. The prompt includes detailed rules and stepwise instructions to tightly control how the model turns JSON into text. Each constraint reduces freedom in style and content so that text-representation is consistent in length, tone, and structure, while still capturing the meaning of each document. This consistency is crucial when these descriptions are later converted to embeddings, since it ensures that differences in vectors reflect differences in meaning, not random variation in wording or format. At the same time, the instructions about semantic distinctiveness, use of concrete values, and avoidance of generic summaries are added to prevent different entities with the same schema from being represented into similar texts.

These four prompts correspond to different points in the design space of $t_1$: Prompt 1 and Prompt 3 are more tightly coupled to the original structure and values, whereas Prompt 2 and Prompt 4 focus on higher-level semantics and human interpretability. In our framework, any of these prompts (or combinations of them) can be used to generate one or more text-representations per JSON document.

## 5. Joining embeddings in a distributed environment

This section focuses on giving an overview of the computation of a distributed computation of the cosine similarity. In particular, once the text-representations $T_1, T_2$ have been mapped to embeddings $\varepsilon_1, \varepsilon_2$, the FLESHJ-Join is reduced to a similarity join over two large collections of $d$-dimensional
vectors. Given $\mathcal{E}_i = \{(id(J_i), \vec{e}_i)|J_i \in C_i \text{ and } \vec{e}_i = t_2(t_1(J_i))\}$, with $i = 1, 2$, we want to find all pairs $(J_1, J_2)$ such that $sim_{cos}(\vec{e}_1, \vec{e}_2) \ge \tau$, for a given similarity threshold $\tau$. A naïve evaluation would compare all pairs, with $\mathcal{O}(|\mathcal{E}_1| \cdot |\mathcal{E}_2|)$ similarity computations, and is infeasible in large-scale settings. In a distributed environment, we also need to minimise communication (shuffle) cost and to balance the computational work across compute nodes. In this section, we describe how the embedding join can be expressed in MapReduce notation and outline several blocking and filtering strategies (e.g., locality-sensitive hashing) that aim to reduce both communication and computation cost. Note that we do not exhaustively investigate the optimization of the distributed evaluation of cosine similarity of large collections of $d$-dimensional vectors, since this is considered a different research topic [40, 41].

As MapReduce formulation, we consider as input to the join a unified multiset of key-value pairs of the form $< id(J), (\vec{e}_J, C_J) >$, where $e_J$ is the document embedding of the JSON document $J \in C_1 \cup C_2$, and $C_J \in \{C_1, C_2\}$. To compute the join, we consider a MapReduce process of two phases (MapReduce jobs). The first phase computes all the pairs $(id(J_1), id(J_2))$ such that $J_1 \in C_1, J_2 \in C_2$, and $sim_{cos}(\vec{e}_{J_1}, \vec{e}_{J_2}) \ge \tau$. The second one simply finds the JSON documents $J_1, J_2$ in the collections $C_1, C_2$, based on their ids, and construct the join output (i.e., $(id(J_1), id(J_2)) \to [J_1, J_2]$). Here, we focus on the first phase. In this context, a generic MapReduce job finding the pairs of document ids that are similar can be expressed as follows:

$$ \begin{aligned} \langle id(J), (\vec{e}_J, C_J) \rangle & \xrightarrow{\text{Map}} \{ \langle k, (id(J), \vec{e}_J, C_J) \rangle | k \in K(J) \} \\ \langle k, \{(id(J), \vec{e}_J, C_J)\} \rangle & \xrightarrow{\text{Reduce}} \{(id(J_1), id(J_2)) | sim_{cos}(\vec{e}_1, \vec{e}_2) \ge \tau\}, \end{aligned} $$

where $K(J)$ is a set of blocking keys that determine to which reducers a document is sent. Each reducer receives all embeddings associated with a given key $k$ and computes a local similarity join between the subsets from $\mathcal{E}_1$ and $\mathcal{E}_2$. Finding a blocking pattern is critical, since it governs the communication cost, via the number of emitted pairs and the replication factor $|K(J)|$. In the following, we outline two main choices for $K(J)$, although additional methods (e.g., Splitting algorithm [7] or FACET algorithm [40]) can be adopted by the relevant literature (e.g., [40, 7, 41]).

Initially, we consider a naive approach (similar to the naive approach presented in [8, 42]), where each embedding is hashed to a set of buckets in an $n \times n$ matrix $M$. In particular, we assign the collections $\mathcal{E}_1$ and $\mathcal{E}_2$ to rows and columns of $M$ (or vice versa), respectively, so that each embedding $e_1 \in \mathcal{E}_1$ (resp. $e_2 \in \mathcal{E}_2$) is hashed to one row (resp. column) of $M$ using a hash function $h_1$ (resp. $h_2$). For each cell $(i, j)$ of $M$, with $0 \le i \le j < n$, a bucket is considered defined by the key $(i, j)$. For each embedding $\vec{e}_{J_1} \in \mathcal{E}_1$ of document $J_1$, then it is emitted to buckets having keys in $K(J_1) = \{(h_1(\vec{e}_{J_1}), j) | 0 \le h_1(\vec{e}_{J_1}) \le j < n\}$. Similarly, for each embedding $\vec{e}_{J_2} \in \mathcal{E}_2$ of document $J_2$, we have $K(J_2) = \{(i, h_2(\vec{e}_{J_2})) | 0 \le i \le h_2(\vec{e}_{J_2}) < n\}$. Then, the reducer simply computes the cosine similarity of each pair of embeddings of $\mathcal{E}_1$ and $\mathcal{E}_2$ received by the reduce task, and returns the pair of ids in case the similarity satisfies the threshold constraint.

As a second approach, motivated by the related literature (e.g., [43]), we use Locality-Sensitive Hashing (LSH) to minimize the communication cost. For cosine similarity, locality-sensitive hash families LSH provide a probabilistic mechanism to group similar vectors in the same buckets [44, 45]. Let $h_\ell$, with $\ell = 1, \dots, L$, be $L$ independent LSH functions mapping $\mathbb{R}^d$ to a discrete signature space (e.g., integer bucket identifiers). For each embedding $\vec{e}_J$ of a JSON document $J$, we define the blocking-key set as $K(J) = \{h_\ell(\vec{e}_J) | \ell = 1, \dots, L\}$. In the Map phase, for each input pair $< id(J), (\vec{e}_J, C_J) >$, the map function emits $L$ key-value pairs of the form $< k, (id(J), \vec{e}_J, C_J) >$, where $k \in K(J)$ (i.e., one for each hash function). In the Reduce phase, for each hash value $k$, the reducer receives two subsets $B_1(k)$ and $B_2(k)$ corresponding to embeddings from $C_1$ and $C_2$, respectively, and computes a local similarity join $B_1(k) \bowtie \tau B_2(k)$ by evaluating cosine similarity only within the bucket, optionally followed by a duplicate-elimination step across buckets. This scheme reduces communication cost because each embedding is replicated only $L$ times, instead of being compared against all vectors on the opposite side, and it reduces computation because each reducer processes only candidate pairs that share at least one hash bucket.
The aforementioned methods typically represent deterministic and probabilistic approaches. In particular, the naive approach guarantees that all the pairs are going to be compared using cosine similarity, but the communication and replication cost is quite high. Hence, if *r* is the number of reducers, defining a matrix *n*×*n*, it is recommended to select *n* ≈ √2* *r*. In contrast, the LSH-based approach trades deterministic coverage for probabilistic guarantees on recall and reduced communication. Intuitively, increasing the parameter *L* reduces the probability of missing a true match, at the cost of replicating each embedding to more buckets.

# 6. Preliminary Evaluation

This section presents a preliminary evaluation of our approach, focusing on assessing the generation of the text-representations from JSON documents. To conduct the experiments, we initially generated a dataset<fnref n="1" /> of around 200K JSON documents through ChatGPT<fnref n="2" />, so that the 5% of these documents are at least 80% similar, taking into account similarity in content and/or structure (i.e., same content in different structure, same content in different structure, or mix of them). We also took into account cases where the documents have the same structure but different content. We then selected a representative sample of around 31 documents (where 9 pairs join with similarity greater than 80%), which were also manually checked to ensure the similarity score estimated by ChatGPT.

To generate the text-representation of each document in the sample, we applied the four prompts presented in Section 4.2 to Meta Llama 3.3-70b-instruct through Openrouter API <fnref n="3" />. Then, we generate the corresponding embeddings using the all-MiniLM-L6-v2 model. The 220 embeddings are compared using cosine similarity, and we evaluated the join result in terms of accuracy, precision, and recall. An overview of the results is illustrated in Figure 5, where the horizontal axis represents the precision, the vertical axis the recall, and the size of each bubble the accuracy score. The scores are also summarized in Table 1.

Analyzing the performance of the prompts, we can see that 3 prompts achieve quite an impressive accuracy score (≥ 95%). Looking at the precision and recall, Prompt 3 is the leader, and Prompt 4 is very close. Note here that to compute the accuracy, precision, and recall scores, we focused on pairs of documents with similarity greater than 80%.

<table><caption>**Table 1**

Accuracy, Precision & Recall scores</caption><thead><tr><td>Prompt</td><td>Precision</td><td>Recall</td><td>Accuracy</td></tr></thead><tbody><tr><td>Prompt 1</td><td>47%</td><td>89%</td><td>95%</td></tr><tr><td>Prompt 2</td><td>13%</td><td>100%</td><td>74%</td></tr><tr><td>Prompt 3</td><td>82%</td><td>100%</td><td>99%</td></tr><tr><td>Prompt 4</td><td>64%</td><td>100%</td><td>98%</td></tr></tbody></table>

# 7. Related Work

Existing work on fuzzy joins in distributed environments has largely focused on structural or token-based similarity measures [46, 47, 48, 8, 49], often combined with optimization techniques such as filtering, prefix indexing or partition-based pruning. Approaches leveraging MapReduce and its derivatives [8, 50, 9] have enabled large-scale execution, with improvements including Bloom filter-based pruning, ball hashing and substring splitting [50] to reduce candidate comparisons. While these methods are effective for structure-oriented matching, for example when using Hamming or edit distance on flattened keys, they struggle to capture cases where two JSON documents are structurally different but

<footnote n="1">¹The dataset is available here:</footnote>

<footnote n="2">²Open AI ChatGPT, https://chatgpt.com/</footnote>

<footnote n="3">³https://openrouter.ai/</footnote>
<figure><img src="image_10.png" /><figcaption>Figure 5: Performance of the Prompts - an overview</figcaption></figure>

semantically equivalent. Recent advances in semantic similarity, such as sentence embeddings and large language models (LLMs), offer new opportunities, but integrating them efficiently into a distributed fuzzy join pipeline introduces new computational and communication bottlenecks. Moreover, research on semantic similarity in order to create clusters of files has been done [51, 52, 53, 54], and shows that semantic similarity between JSON files is already useful for clustering; we therefore investigate how to repurpose these semantic signals to perform fuzzy joins with higher recall and precision.

Computing such a join is non-trivial, especially when performed at scale. In addition to the inherent complexity of comparing high-dimensional semantic embeddings, the integration of language models into a distributed computation pipeline introduces additional challenges — including network overhead, fault tolerance, load balancing, and the serialization of model calls across the cluster. These factors make the deployment of fuzzy join operations in real-world distributed systems particularly demanding.

## 8. Conclusion and Future Work

In this work, we propose an approach to compute the fuzzy join of collections of JSON documents, based on semantic similarity, demonstrating how lightweight description strategies and embedding-based techniques can provide meaningful matching beyond exact structural correspondence. To compute the semantic similarity, we use 4 different prompts to generate the text-representations of each document in the joining collections, focusing on preserving the structure and the content of each document. The text-representations are then transformed to embeddings which are compared through a distributed framework (e.g., MapReduce or Spark) to achieve a scalable join computation. Our approach ensures scalable computation of the fuzzy join, taking into account arbitrary JSON structures and content.

As future steps, we aim to further investigate the prompt design, as well as focus on distributed computation of the embedding generation. We also plan to extensively evaluate our approach through large datasets and investigate further optimizations, in terms of both overall computational efficiency and accuracy.
Declaration on Generative AI

During the preparation of this work, the authors used GPT in order to: Grammar and spelling check.
After using these tool/service, the authors reviewed and edited the content as needed and take full
responsibility for the publication's content.

References

[1] Y. Jiang, G. Li, J. Feng, W.-S. Li, String similarity joins: An experimental evaluation, Proc. VLDB Endow. 7 (2014) 625–636.

[2] R. Vernica, M. J. Carey, C. Li, Efficient Parallel Set-similarity Joins Using MapReduce, in: SIGMOD, 2010, pp. 495–506.

[3] D. Deng, G. Li, S. Hao, J. Wang, J. Feng, Massjoin: A mapreduce-based method for scalable string similarity joins, in: ICDE, 2014, pp. 340–351.

[4] X. Wang, D. Sun, Qjoin: A q-sample-based method for large-scale string similarity joins, in: ISCID, 2018, pp. 45–48.

[5] T.-T.-Q. Tran, T.-C. Phan, A. Laurent, L. d' Orazio, Improving hamming distance-based fuzzy join in mapreduce using bloom filters, in: FUZZ-IEEE, 2018, pp. 1–7.

[6] T. Thi-To-Quyen, P. Thuong-Cang, A. Laurent, L. d'Orazio, Optimization for large-scale fuzzy joins using fuzzy filters in mapreduce, in: FUZZ-IEEE, IEEE, 2020, pp. 1–8.

[7] F. N. Afrati, A. D. Sarma, D. Menestrina, A. Parameswaran, J. D. Ullman, Fuzzy Joins Using MapReduce, in: ICDE, 2012, pp. 498–509.

[8] R. Uhartegaray, L. D'Orazio, M. Damigos, E. Kalogeros, Scalable computation of fuzzy joins over large collections of json data, IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (2023).

[9] A. Drissi, L. d'Orazio, M. Damigos, Scalable structural similarity analysis of json documents using mapreduce, IEEE International Conference on Fuzzy Systems, Jul 2025, Reims (51), France (2025).

[10] T. Bray, The javascript object notation (JSON) data interchange format, RFC 7158 (2014) 1–16.

[11] S. Gössner, Jsonpath-xpath for json, URL http://goessner.net/articles/JsonPath (2007) 48.

[12] P. Bourhis, J. L. Reutter, D. Vrgoč, JSON: Data model and query languages, Information Systems 89 (2020) 101478.

[13] J. Dean, S. Ghemawat, Mapreduce: Simplified data processing on large clusters, in: Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI), San Francisco, CA, USA, 2004, pp. 137–150.

[14] W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, et al., A survey of large language models, arXiv preprint arXiv:2303.18223 1 (2023).

[15] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, Advances in neural information processing systems 30 (2017).

[16] A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, et al., Improving language understanding by generative pre-training (2018).

[17] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al., Llama: Open and efficient foundation language models, arXiv preprint arXiv:2302.13971 (2023).

[18] P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, G. Neubig, Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing, ACM computing surveys 55 (2023) 1–35.

[19] J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, Q. V. Le, Finetuned language models are zero-shot learners, arXiv preprint arXiv:2109.01652 (2021).

[20] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al., Chain-of-thought prompting elicits reasoning in large language models, Advances in neural information processing systems 35 (2022) 24824–24837.
[21] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, et al., Retrieval-augmented generation for knowledge-intensive nlp tasks, Advances in neural information processing systems 33 (2020) 9459-9474.

[22] Y. Bengio, R. Ducharme, P. Vincent, C. Jauvin, A neural probabilistic language model, Journal of machine learning research 3 (2003) 1137-1155.

[23] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, Advances in neural information processing systems 26 (2013).

[24] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 2019, pp. 4171-4186.

[25] Z. Lin, M. Feng, C. N. d. Santos, M. Yu, B. Xiang, B. Zhou, Y. Bengio, A structured self-attentive sentence embedding, arXiv preprint arXiv:1703.03130 (2017).

[26] N. Reimers, I. Gurevych, Sentence-bert: Sentence embeddings using siamese bert-networks, arXiv preprint arXiv:1908.10084 (2019).

[27] M. Yu, G. Li, D. Deng, J. Feng, String similarity search and join: a survey, Frontiers of Computer Science 10 (2016) 399-417.

[28] W. H. Gomaa, A. A. Fahmy, et al., A survey of text similarity approaches, international journal of Computer Applications 68 (2013) 13-18.

[29] J. Tekli, R. Chbeir, K. Yetongnon, An overview on xml similarity: Background, current trends and future directions, Computer science review 3 (2009) 151-173.

[30] G. Cobena, S. Abiteboul, A. Marian, Detecting changes in xml documents, in: ICDE, IEEE, 2002, pp. 41-52.

[31] C. Donnat, S. Holmes, Tracking network dynamics: A survey of distances and similarity metrics, arXiv preprint arXiv:1801.07351 (2018).

[32] D. Koutra, A. Parikh, A. Ramdas, J. Xiang, Algorithms for graph similarity and subgraph matching, in: Proc. Ecol. inference conf, volume 17, Citeseer, 2011.

[33] M. Pawlik, N. Augsten, Tree edit distance: Robust and memory-efficient, Information Systems 56 (2016) 157-173.

[34] R. Yang, P. Kalnis, A. K. Tung, Similarity evaluation on tree-structured data, in: SIGMOD, 2005, pp. 754-765.

[35] L. Wang, K. Zhang, Space efficient algorithms for ordered tree comparison, Algorithmica 51 (2008) 283-297.

[36] D. U. Priya, P. S. Thilagam, Json document clustering based on schema embeddings, Journal of Information Science (2022) 0165551522116522.

[37] T. Hütter, N. Augsten, C. M. Kirsch, M. J. Carey, C. Li, JEDI: These aren't the JSON documents you're looking for?, in: SIGMOD, 2022, pp. 1584-1597.

[38] E. Ben Hadj Yahia, J.-R. Falleri, L. Réveillère, Polly: A language-based approach for custom change detection of web service data, in: ICSOC, Springer, 2017, pp. 430-444.

[39] H. Cao, J.-R. Falleri, X. Blanc, L. Zhang, Json patch for turning a pull rest api into a push, in: ICSOC, Springer, 2016, pp. 435-449.

[40] B. Yang, H. J. Kim, J. Shim, D. Lee, S.-g. Lee, Fast and scalable vector similarity joins with mapreduce, Journal of Intelligent Information Systems 46 (2016) 473-497.

[41] R. Baraglia, G. D. F. Morales, C. Lucchese, Document similarity self-join with mapreduce, in: 2010 IEEE International Conference on data mining, IEEE, 2010, pp. 731-736.

[42] A. Okcan, M. Riedewald, Processing theta-joins using mapreduce, in: Proceedings of the 2011 ACM SIGMOD International Conference on Management of data, 2011, pp. 949-960.

[43] X. Hu, Y. Tao, K. Yi, Output-optimal parallel algorithms for similarity joins, in: Proceedings of the 36th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, 2017, pp. 79-90.

[44] M. S. Charikar, Similarity estimation techniques from rounding algorithms, in: Proceedings of the thiry-fourth annual ACM symposium on Theory of computing, 2002, pp. 380-388.
[45] A. Andoni, P. Indyk, T. Laarhoven, I. Razenshteyn, L. Schmidt, Practical and optimal lsh for angular distance, Advances in neural information processing systems 28 (2015).

[46] T. Hütter, N. Augsten, C. M. Kirsch, M. J. Carey, C. Li, Jedi: These aren't the json documents you're looking for..., SIGMOD '22 (2022).

[47] A. Sevim, A. Eldawy, E. P. Carman, M. J. Carey, V. J. Tsotras, Fudj: Flexible user-defined distributed joins, IEEE International Conference on Fuzzy Systems (2024).

[48] T. Mizokami, S. Bou, T. Amagasa, Subtree similarity search based on structure and text, in: Big Data Analytics and Knowledge Discovery, 2024.

[49] M. Yu, G. Li, D. Deng, J. Feng, String similarity search and join : A survey, Frontiers of Computer Science (2015).

[50] T.-T.-Q. Tran, T.-C. Phan, A. Laurent, L. D'Orazio, Optimization for large-scale fuzzy joins using fuzzy filters in mapreduce, IEEE International Conference on Fuzzy Systems (2020).

[51] U. P. Dharmaraj, P. S. Thilagam, Json document clustering based on schema embeddings, Journal of information science (2022).

[52] U. P. Dharmaraj, P. S. Thilagam, Json document clustering based on structural similarity and semantic fusion, in: Proceedings of International Conference on Computational Intelligence and Data Engineering, 2023.

[53] D. Chandrasekaran, V. Mago, Evolution of semantic similarity - a survey, ACM Computing Surveys (2021).

[54] D. R. Santana, P. H. S. Lima, L. A. Ribeiro, Em-join: Efficient entity matching using embedding-based similarity join, in: In Proceedings of the 27th International Conference on Enterprise Information Systems, 2025.