## 3 Related Works
When the packet includes broken loading data, you need to track the problem, troubleshoot and quickly resend the corrected data [14]. Limitations of classic ETL process applications highlight the necessity for process modifications to leverage all the power of GenAI during all data cycles, to be able to process data of different formats and handle both batch and streaming data. It is important to highlight the Medallion architecture, a data engineering framework that organizes data into multiple layers or \"medals\"--Bronze, Silver, and Gold--each representing a different stage of data refinement [15]. In this structure, raw data are initially ingested into the Bronze layer, where it is stored in its most unprocessed form. The data are then cleaned, filtered, and enriched in the Silver layer, making it more structured and reliable. Finally, in the Gold layer, the data undergoes further refinement and aggregation, preparing it for high-quality business analytics, reporting, and advanced use cases such as machine learning. Delta Lake, often associated with the Medallion architecture, is an open-source storage layer designed to enhance the reliability and performance of data lakes [16]. It offers features like ACID transactions, scalable metadata handling, and the ability to efficiently manage both batch and streaming data. ACID transactions, which stand for atomicity, consistency, isolation, and durability, ensure the reliable processing of database transactions, particularly in relational databases. However, in non-relational or distributed databases, the strict enforcement of these properties can create performance bottlenecks [17]. Moreover, while Delta Lake introduces robustness, it also adds complexity, particularly in managing the transaction log and maintaining ACID properties across distributed systems [16]. The continuous need to update the transaction log and ensure data consistency across large datasets can slow down data processing, especially in real-time scenarios. Additionally, the requirement to store multiple versions of data, driven by features like time travel and versioning, can lead to increased storage costs. [18]. The EGTL model represents a significant advancement in data integration methodologies, establishing itself as an innovative approach within the broader research landscape. Traditional ETL processes have long served as the foundation of data warehousing and business intelligence, primarily focusing on structured data and batch processing. However, the rise of big data [19], the demand for real-time analytics, and the increasing complexity of data sources have revealed the limitations of these traditional approaches [20]. By incorporating GenAI into the data pipeline, the EGTL model addresses these challenges, enhancing data quality, enabling the extraction of novel insights, and offering greater flexibility in data processing. It introduces previously unavailable opportunities, such as generating code based on extracted data and dynamically creating new pages or modules through continuous self-improvement. This approach aligns with emerging trends in data engineering, including the adoption of DataOps and data mesh principles, which prioritize automation, continuous integration, and decentralized data management. The EGTL model's relevance is further underscored by its ability to bridge the gap between conventional ETL frameworks and modern, AI-driven data environments, making it a valuable contribution to the ongoing evolution of data processing techniques.