Package: dbt-risingwave
dbt-risingwaveARisingWaveadapter plugin fordbt.RisingWaveis a cloud-native streaming database that uses SQL as the interface language. It is designed to reduce the complexity and cost of building real-time applications.https://www.risingwave.comdbtenables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.Use dbt for data transformations in RisingWaveGetting startedThe package has not been published to PyPI, please install it via git.Installdbt-risingwavepython3-mpipinstalldbt-risingwaveGetRisingWaverunningPlease followthisguide to setup a functional RisingWave instance.Configuredbtprofile fileThe profile file is located in~/.dbt/profiles.yml. Here's an example of how to use it with RisingWave.default:outputs:dev:type:risingwavehost:127.0.0.1user:rootpass:""dbname:devport:4566schema:publictarget:devRundbt debugto check whether configuration is correct.ModelsThe dbt models for managing data transformations in RisingWave is similar to typical dbt sql models. The main differences are the materializations. We customized the materializations to fit the data processing model of RisingWave.MaterializationsINFOmaterialized_viewCreate a materialized view. This materialization is corresponding to the incremental one in dbt. To use this materialization, add {{ config(materialized='materialized_view') }} to your model SQL files.materializedview(Deprecated) only for backward compatibility, usematerialized_viewinsteadephemeralThis materialization uses common table expressions in RisingWave under the hood. To use this materialization, add {{ config(materialized='ephemeral') }} to your model SQL files.tableCreate a table. To use this materialization, add {{ config(materialized='table') }} to your model SQL files.viewCreate a view. To use this materialization, add {{ config(materialized='view') }} to your model SQL files.incrementalUsematerialized_viewinstead. Since RisingWave is designed to use materialized view to manage data transformation in an incremental way, you don’t need to use the incremental materialization and can just use materializedview.sourceDefine a source {{ config(materialized='source') }}. You need to provide your create source statement as a whole in this model.table_with_connectorDefine a table with a connector {{ config(materialized='table_with_connector') }}. You need to provide your create table with connector statement as a whole in this model. Because dbttablehas its own semantics, RisingWave usetable_with_connectorto distinguish itself from it.sinkDefine a sink {{ config(materialized='sink') }}. You need to provide your create sink statement as a whole in this model.To learn how to use, you can check RisingWave offical exampledbt_rw_nexmark.DBT RUN behaviordbt run: only create new models (if not exists) without dropping any models.dbt run --full-refresh: drop models and create the new ones. This command can make sure your streaming pipelines are consistent with what you define in dbt models.Graph operatorsGraph operatorsis useful when you want to only recreate a subset of your models.dbtrun--select"my_model+"# select my_model and all childrendbtrun--select"+my_model"# select my_model and all parentsdbtrun--select"+my_model+"# select my_model, and all of its parents and childrenTestsAll items below have been tested against the the latest RisingWave daily build verison.Offical examplejaffle_shopis tested.RisingWave offical exampledbt_rw_nexmarkis tested.