How should we use data integration frameworks to break the dilemma of full lifecycle implementation of data services?
Published: 2022-05-07 10:49:23


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ou658vk1tc024wgwbbgu.jpg)
As we enter the digital economy, both companies and engineers are acutely aware of the increasing importance of data in production. Big data, known as the “oil” of the digital age, holds tremendous value and possibilities that are waiting to be explored and utilized.

As we all know, the life cycle of data is divided into stages such as collection, storage, integration, analysis and application, archiving, and destruction. Data integration is a crucial process of the data lifecycle, which is the determinant to whether the enterprise can get the most value from the data center with the least cost.

As data resources become an essential element of production, development, and even survival, how can data integration help enterprises implement the full lifecycle of data services? On May 14, we will be joined by technical experts and open source contributors from Apache SeaTunnel(Incubating) and Apache InLong(Incubating) to talk about their practical experiences and insights on this topic.

In this Meetup themed on “Data Service Full Lifecycle Implementation Practice”, senior engineers from enterprises such as Yoozoo, Tencent, and Whaleops will talk about the design and implementations in the network data access platforms, the improvement of message-oriented middlewares, API refactoring and the implementation and optimization of the PCG big data support system, and other topics based on Apache SeaTunnel (Incubating) and Apache InLong (Incubating), with the difficulties and gains encountered in their development practice as a blueprint, and recommend the corresponding solutions and summaries.

Apache SeaTunnel (Incubating) and Apache InLong (Incubating) are both excellent projects in the Apache open-source community. As the next-generation, high-performance, distributed, mass data integration framework, Apache SeaTunnel (Incubating) is dedicated to reducing learning costs and accelerating the deployment of distributed data processing capabilities in production environments. While Apache InLong (Incubating) is a one-stop integration framework for massive amounts of data, providing automated, secure, and reliable data transfer capabilities.

What will happen between these two excellent open-source products in the data services space? Join us at the Apache SeaTunnel (Incubating) & Apache InLong (Incubating) Joint Meetup online on May 14 to interact with the speakers. Your questions will be answered here!

## 01 Registration

The registration for the Apache SeaTunnel(Incubating) & Apache InLong(Incubating) Joint Meetup | May 14 is open, register for free now!

Time: 2022–5–14 14:00–17:00

Format: Online meetup

Click on the link to register for free: https://www.slidestalk.com/m/777

Join the Slack group: https://join.slack.com/t/apacheseatunnel/shared_invite/zt-18lmwijxg-kYXQTOuDBBzfXR64nlPj9A


## 02 Schedule


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/drxuutzatiipcol5h094.png)


Xiaodong Zhang, Big Data Development Engineer | YOOZOO

Responsible for the data platform construction of the data center, mainly the development of the data access platform.

Time: 14:05–14:50

Speech Topic: Design and Implementation of Apache SeaTunnel(Incubating) in YOOZOO Data Access Platform

Introduce how to integrate Apache SeaTunnel(Incubating) into the data access platform and the achievement.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kct36cwfvexa3xvrrson.png)


Guocheng Zhang, Senior Engineer | Tencent Data Platform Department

Responsible for the development of the data access platform

Time: 14:50–15:20

Speech Topic: Improvement of Apache InLong (Incubating) on ​​message middleware

Introduce the challenges encountered by the data access platform when the average daily message access exceeds one trillion pieces, the improvement plans, and effects.

**15:20–15:30 Lucky Draw Time**


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1lgh8ntvl7cg5s2udk60.png)


Li Zongwen, Senior Engineer, Apache SeaTunnel(Incubating) & Flink Contributor, Flink CDC & Debezium Contributor | WhaleOps

Time: 15:30–16:15

Speech Topic: Apache SeaTunnel(Incubating) API refactoring

Why do we need to refactor the Apache SeaTunnel(Incubating) API? How to refactor Apache SeaTunnel(Incubating) API?


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xa86zbhq641skgm856pg.png)


Lu Chunliang, Senior Engineer | Tencent PCG

Responsible for the development of the basic big data support system for Tencent PCG big data collection, reporting, transmission, and distribution.

Time: 16:15–17:00

Speech Topic: Implementation and optimization of Apache InLong (Incubating) in Tencent PCG big data support system

Introduce Apache InLong(Incubating)’s protection and automation measures to deal with abnormal failures and traffic shocks, reduce loss and repetition rate, and control O&M investment.

The Apache SeaTunnel(Incubating) & Apache InLong(Incubating) Joint Meetup will be online on May 14 at 14:00, and we look forward to seeing you later!


## About SeaTunnel


SeaTunnel (formerly Waterdrop) is an easy-to-use, ultra-high-performance distributed data integration platform that supports real-time synchronization of massive amounts of data and can synchronize hundreds of billions of data per day in a stable and efficient manner.


Why do we need SeaTunnel?



SeaTunnel does everything it can to solve the problems you may encounter in synchronizing massive amounts of data.


- Data loss and duplication
- Task buildup and latency
- Low throughput
- Long application-to-production cycle time
- Lack of application status monitoring

SeaTunnel Usage Scenarios

- Massive data synchronization
- Massive data integration
- ETL of large volumes of data
- Massive data aggregation
- Multi-source data processing

Features of SeaTunnel

- Rich components
- High scalability
- Easy to use
- Mature and stable

How to get started with SeaTunnel quickly?

Want to experience SeaTunnel quickly? SeaTunnel 2.1.0 takes 10 seconds to get you up and running.

https://seatunnel.apache.org/docs/2.1.0/developement/setup

How can I contribute?

We invite all partners who are interested in making local open-source global to join the SeaTunnel contributors family and foster open-source together!

Submit an issue:

https://github.com/apache/incubator-seatunnel/issues

Contribute code to:

https://github.com/apache/incubator-seatunnel/pulls

Subscribe to the community development mailing list :

dev-subscribe@seatunnel.apache.org

Development Mailing List :

dev@seatunnel.apache.org

Join Slack:

https://join.slack.com/t/apacheseatunnel/shared_invite/zt-10u1eujlc-g4E~ppbinD0oKpGeoo_dAw

Follow Twitter:

https://twitter.com/ASFSeaTunnel

Come and join us!