# Security and Ownership in User-Defined Data Meshes
## 2 Technical Background  
### Understanding Data Lakes and Data Meshes  
A data lake (DL) is a centralized architecture designed to store vast amounts of structured, unstructured, and semi-structured data on any scale. Unlike traditional databases or data warehouses that require data to be structured before storage, a DL can host raw data in its native format [13]. This means that data from various sources such as logs, clickstreams, social media, videos, and sensors can be stored without the need for pre-defined schemas. DLs offer storage flexibility, allowing the storage of data in its raw form without the definition of upfront schema. This feature enables the accommodation of various data types and formats from diverse sources at any production frequency. DLs are highly scalable and capable of handling very large datasets making them ideal for big data applications. They often provide cost-effective storage options by leveraging cloud object storage, resulting in more economical solutions compared to traditional data warehouses. DLs seamlessly integrate with tools and technologies that enable the stored data to be processed, queried, and analyzed. Properly configured DLs can implement security measures and data governance policies to ensure privacy and compliance with regulations. While DLs offer a high degree of flexibility, they require careful management to prevent them from becoming data swamps, that is, hosting places where data are poorly organized, difficult to find, and hard to analyze [9]. To address this concern, practices like metadata management, data cataloguing, and the establishment of data governance policies are crucial. Figure 1 presents the structure of a DL and an algorithmic description of how the DL concept works in practice, from collecting data, annotating data using metadata, storing data, and finally retrieving data based on metadata tags. The concept of a data mesh (DM), as mentioned in the previous section, was introduced in 2019 [14], which essentially represents a novel approach to data management within large organizations. Unlike traditional methods, a DM emphasizes several key concepts to revolutionize data handling. Firstly, it advocates for domain-oriented ownership. This means that data domains are entrusted to the teams or business units possessing the highest expertise in that specific domain. These teams bear the responsibility for ensuring the quality, accessibility, and privacy of their respective domain's data. Additionally, a DM promotes the idea of decentralized data products. Here, data are treated as a product and each domain team is accountable for the entire data lifecycle within their domain. This encompasses tasks such as production, consumption, quality assurance, privacy measures, and comprehensive documentation. Furthermore, DMs advocate for federated computational governance, an approach where each domain team defines and enforces the computational logic specific to that domain. This logic is then executed within the broader context of the mesh [15]. To facilitate autonomy and efficiency, DM incorporates a self-serving data infrastructure. This infrastructure is designed to empower domain teams with the necessary tools and resources to independently manage their data products, reducing the reliance on centralized data engineering teams. Embracing an API-first approach, DM encourages the utilization of Application Programming Interfaces (APIs) for seamless data exchange and communication between different components of the system. This promotes loose coupling and flexibility in how data are consumed and utilized. Figure 2 presents the structure of a DM and the algorithm that serves as the core of its operation. Furthermore, DMs emphasize a holistic view of the data product lifecycle. This encompasses stages such as discovery, ingestion, processing, storage, access, and consumption. Each of these stages is carefully considered and managed by the respective domain teams,  
Figure 1: Data lake architecture and the concept algorithmic approach.