Package: dbt-greenplum
dbtenables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.dbt is the T in ELT. Organize, cleanse, denormalize, filter, rename, and pre-aggregate the raw data in your warehouse so that it's ready for analysis.dbt-greenplumThedbt-greenplumpackage contains the code enabling dbt to work with Greenplum. This adapter based onpostgres-adapterwith a bit difference for a greenplum specific featuresInstallationEasiest way to start use dbt-greenplum is to install it using pippip install dbt-greenplum==<version>Where<version>is same as your dbt versionAvailable versions:0.19.21.0.41.2.01.4.01.5.0Supported FeaturesYou can specify following settings:Storage typeheapappendoptimizedDistributiondistributed randomlyby defautdistributed by (column, [ ... ] )by setting updistributed_byparameter in the model configdistributed replicatedby setting updistributed_replicated=trueparameter in the model configTable orientationorientation=columby defaultorientation=rowby setting uporientationparameter inrowin the model configCompress type, level and blocksize with default valuescompresstype=ZLIB,compresslevel=1,blocksize=32768You can also specifyblocksize,compresstype,compresslevelin the model configappendoptimizedpreference by default istrue, also you can override it by setting upappendoptimizedfield in the model configPartitions (see "Partition" chapter below)Heap table exampleTo create heap table setappendoptimizedparameter value tofalse{{config(...materialized='table',appendoptimized=false...)}}select1as"id"will produce following SQL codecreatetable"<db_name>"."<schema_name>"."<table_name>"with(appendoptimized=false)as(select1as"id")DISTRIBUTEDRANDOMLY;Appendoptimized table exampleYou can useappendopimizedorappendonly(legacy) to create appendoptimized tableSuch model definition{{config(materialized='table',distributed_by='id',appendoptimized=true,orientation='column',compresstype='ZLIB',compresslevel=1,blocksize=32768)}}withsource_dataas(select1asidunionallselectnullasid)select*fromsource_datawill produce following sql codecreatetable"dvault"."dv"."my_first_dbt_model__dbt_tmp"with(appendoptimized=true,blocksize=32768,orientation=column,compresstype=ZLIB,compresslevel=1)as(withsource_dataas(select1asidunionallselectnullasid)select*fromsource_data)distributedby(id);altertable"dvault"."dv"."my_first_dbt_model__dbt_tmp"renameto"my_first_dbt_model";PartitionsGreenplum does not support partitions withcreate table asconstruction, so you need to build model in two stepscreate table schemainsert dataTo implement partitions into you dbt-model you need to specify on of the following config parameters:fields_string- definition of columns name, type and constraintsone of following way to configure partitionsraw_partitionby defaultpartition_type,partition_column,partition_specpartition_type,partition_column,partition_start,partition_end,partition_everypartition_type,partition_column,partition_valuesdefault_partition_name- name of default partition 'other' by defaultLet consider examples of definition model with partitionsusingraw_partitionparameter{%setfields_string%}idint4null,incomingdatetimestampNULL{%endset%}{%setraw_partition%}PARTITIONBYRANGE(incomingdate)(START('2021-01-01'::timestamp)INCLUSIVEEND('2023-01-01'::timestamp)EXCLUSIVEEVERY(INTERVAL'1 day'),DEFAULTPARTITIONextra);{%endset%}{{config(materialized='table',distributed_by='id',appendoptimized=true,orientation='column',compresstype='ZLIB',compresslevel=1,blocksize=32768,fields_string=fields_string,raw_partition=raw_partition,default_partition_name='other_data')}}withsource_dataas(select1asid,'2022-02-22'::timestampasincomingdateunionallselectnullasid,'2022-02-25'::timestampasincomingdate)select*fromsource_datawill produce following sql codecreatetableifnotexists"database"."schema"."my_first_dbt_model__dbt_tmp"(idint4null,incomingdatetimestampNULL)with(appendoptimized=true,blocksize=32768,orientation=column,compresstype=ZLIB,compresslevel=1)DISTRIBUTEDBY(id)PARTITIONBYRANGE(incomingdate)(START('2021-01-01'::timestamp)INCLUSIVEEND('2023-01-01'::timestamp)EXCLUSIVEEVERY(INTERVAL'1 day'),DEFAULTPARTITIONextra);insertinto"database"."schema"."my_first_dbt_model__dbt_tmp"(withsource_dataas(select1asid,'2022-02-22'::timestampasincomingdateunionallselectnullasid,'2022-02-25'::timestampasincomingdate)select*fromsource_data);altertable"dvault"."dv"."my_first_dbt_model"renameto"my_first_dbt_model__dbt_backup";droptableifexists"dvault"."dv"."my_first_dbt_model__dbt_backup"cascade;altertable"database"."schema"."my_first_dbt_model__dbt_tmp"renameto"my_first_dbt_model";Same result you can get usingpartition_type,partition_column,partition_specparameters{%setfields_string%}idint4null,incomingdatetimestampNULL{%endset%}{%-setpartition_type='RANGE'-%}{%-setpartition_column='incomingdate'-%}{%setpartition_spec%}START('2021-01-01'::timestamp)INCLUSIVEEND('2023-01-01'::timestamp)EXCLUSIVEEVERY(INTERVAL'1 day'),DEFAULTPARTITIONextra{%endset%}{{config(materialized='table',distributed_by='id',appendoptimized=true,orientation='column',compresstype='ZLIB',compresslevel=1,blocksize=32768,fields_string=fields_string,partition_type=partition_type,partition_column=partition_column,partition_spec=partition_spec,default_partition_name='other_data')}}withsource_dataas(select1asid,'2022-02-22'::timestampasincomingdateunionallselectnullasid,'2022-02-25'::timestampasincomingdate)select*fromsource_dataalso, you can use third way{%setfields_string%}idint4null,incomingdatetimestampNULL{%endset%}{%-setpartition_type='RANGE'-%}{%-setpartition_column='incomingdate'-%}{%-setpartition_start="'2021-01-01'::timestamp"-%}{%-setpartition_end="'2022-01-01'::timestamp"-%}{%-setpartition_every='1 day'-%}{{config(materialized='table',distributed_by='id',appendoptimized=true,orientation='column',compresstype='ZLIB',compresslevel=1,blocksize=32768,fields_string=fields_string,partition_type=partition_type,partition_column=partition_column,partition_start=partition_start,partition_end=partition_end,partition_every=partition_every,default_partition_name='other_data')}}withsource_dataas(select1asid,'2022-02-22'::timestampasincomingdateunionallselectnullasid,'2022-02-25'::timestampasincomingdate)select*fromsource_dataexample of partition_typeLISTis coming soonTable partition hintsToo check generate sql script use-doption:dbt -d run <...> -m <models>If you want implement complex partition logic with subpartition or something else useraw_partitionparameterGetting startedInstall dbtRead theintroductionandviewpointJoin the dbt CommunityBe part of the conversation in thedbt Community SlackRead more on thedbt Community DiscourseReporting bugs and contributing codeWant to report a bug or request a feature? Let us know onSlack, or openan issueWant to help us build dbt? Check out theContributing GuideCode of ConductEveryone interacting in the dbt project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow thedbt Code of Conduct.