Package: dbt-teradata
dbt-teradataThe dbt Teradata adapter lets you usedbtwith Teradata Vantage.NOTE:This adapter is maintained by Teradata. We are accelerating our release cadence. Starting October 1st, 2023, we will releasedbt-teradatawithin 4 weeks of a minor release or within 8 weeks of a major release ofdbt-core.Installationpip install dbt-teradataIf you are new to dbt on Teradata seedbt with Teradata Vantage tutorial.NOTE:If the virtual environment in Python is not activating properly on Windows, you can try running the below command in the command-line interface (CLI) before attempting to activate the virtual environment.Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUseSample profileHere is a working example of adbt-teradataprofile:my-teradata-db-profile:target:devoutputs:dev:type:teradatahost:localhostuser:dbcpassword:dbcschema:dbt_testtmode:ANSIAt a minimum, you need to specifyhost,user,password,schema(database),tmode.Python compatibilityPlugin versionPython 3.6Python 3.7Python 3.8Python 3.9Python 3.10Python 3.110.19.0.x✅✅✅❌❌❌0.20.0.x✅✅✅✅❌❌0.21.1.x✅✅✅✅❌❌1.0.0.x❌✅✅✅❌❌1.1.x.x❌✅✅✅✅❌1.2.x.x❌✅✅✅✅❌1.3.x.x❌✅✅✅✅❌1.4.x.x❌✅✅✅✅✅1.5.x❌✅✅✅✅✅1.6.x❌❌✅✅✅✅1.7.x❌❌✅✅✅✅dbt dependent packages version compatibilitydbt-teradatadbt-coredbt-teradata-utildbt-util1.2.x1.2.x0.1.00.9.x or below1.6.71.6.71.1.11.1.11.7.x1.7.x1.1.11.1.1Optional profile configurationsLogmechThe logon mechanism for Teradata jobs that dbt executes can be configured with thelogmechconfiguration in your Teradata profile. Thelogmechfield can be set to:TD2,LDAP,KRB5,TDNEGO. For more information on authentication options, go toTeradata Vantage authentication documentation.my-teradata-db-profile:target:devoutputs:dev:type:teradatahost:<host>user:<user>password:<password>schema:dbt_testtmode:ANSIlogmech:LDAPLogdataThe logon mechanism for Teradata jobs that dbt executes can be configured with thelogdataconfiguration in your Teradata profile. Addtional data like secure token, distinguished Name, or a domain/realm name can be set in your Teradata profile usinglogdata. Thelogdatafield can be set to:JWT,LDAP,KRB5,TDNEGO.logdatais not used with the TD2 mechanism.my-teradata-db-profile:target:devoutputs:dev:type:teradatahost:<host>schema:dbt_testtmode:ANSIlogmech:LDAPlogdata:'authcid=usernamepassword=password'port:<port>For more information on authentication options, go toTeradata Vantage authentication documentationStored Password ProtectionStored Password Protection enables an application to provide a connection password in encrypted form to the driver. The plugin supports Stored Password Protection feature through prefixENCRYPTED_PASSWORD(either inpasswordconnection parameter  or inlogdataconnection parameter.passwordmy-teradata-db-profile:target:devoutputs:dev:type:teradatahost:<host>user:<user>password:ENCRYPTED_PASSWORD(file:PasswordEncryptionKeyFileName,file:EncryptedPasswordFileName)schema:dbt_testtmode:ANSIport:<port>logdatamy-teradata-db-profile:target:devoutputs:dev:type:teradatahost:<host>schema:dbt_testtmode:ANSIlogmech:LDAPlogdata:'authcid=usernamepassword=ENCRYPTED_PASSWORD(file:PasswordEncryptionKeyFileName,file:EncryptedPasswordFileName)'port:<port>For full description of Stored Password Protection seehttps://github.com/Teradata/python-driver#StoredPasswordProtection.PortIf your Teradata database runs on port different than the default (1025), you can specify a custom port in your dbt profile usingportconfiguration.my-teradata-db-profile:target:devoutputs:dev:type:teradatahost:<host>user:<user>password:<password>schema:dbt_testtmode:ANSIport:<port>RetriesAllows an adapter to automatically try again when the attempt to open a new connection on the database has a transient, infrequent error. This option can be set using theretriesconfiguration. Default value is 0. The default wait period between connection attempts is one second.retry_timeout(seconds) option allows us to adjust this waiting period.Ifretriesis set to 3, the adapter will try to establish a new connection three times if an error occurs.my-teradata-db-profile:target:devoutputs:dev:type:teradatahost:<host>user:<user>password:<password>schema:dbt_testtmode:ANSIretries:3retry_timeout:10Other Teradata connection parametersThe plugin also supports the following Teradata connection parameters:accountcolumn_namecopcoplastencryptdatafake_result_setsfield_quotefield_seplob_supportloglogdatamax_message_bodypartitionsip_supportteradata_valuessslmodesslcasslcapathsslcrcsslciphersslprotocolbrowserbrowser_tab_timeoutbrowser_timeoutsp_splsessionsrunstartuplogon_timeouthttps_portconnect_timeoutrequest_timeoutFor full description of the connection parameters seehttps://github.com/Teradata/python-driver#connection-parameters.Supported FeaturesMaterializationsviewtableephemeralincrementalIncremental MaterializationThe following incremental materialization strategies are supported:append(default)delete+insertmergeTo learn more about dbt incremental strategies please checkthe dbt incremental strategy documentation.CommandsAll dbt commands are supported.Custom configurationsGeneralEnable view column types in docs-  Teradata Vantage has a dbscontrol configuration flag calledDisableQVCI(QVCI - Queryable View Column Index). This flag instructs the database to buildDBC.ColumnsJQVwith view column type definitions.:information_source: Existing customers, please seeKB0022230for more information about enabling QVCI.To enable this functionality you need to:Enable QVCI mode in Vantage. Usedbscontrolutility and then restart Teradata. Run these commands as a privileged user on a Teradata node:# option 551 is DisableQVCI. Setting it to false enables QVCI.dbscontrol<< EOFM internal 551=falseWEOF# restart Teradatatpareset-yEnableQVCIInstructdbtto useQVCImode. Include the following variable in yourdbt_project.yml:vars:use_qvci:trueFor example configuration, seetest/catalog/with_qvci/dbt_project.yml.ModelsTableThe following options apply to table, snapshots and seed materializations.table_kind- define the table kind. Legal values areMULTISET(default for ANSI transaction mode required bydbt-teradata) andSET, e.g.:in sql materialization definition file:{{config(materialized="table",table_kind="SET")}}in seed configuration:seeds:<project-name>:table_kind:"SET"For details, seeCREATE TABLE documentation.table_option- define table options. Legal values are:{MAP=map_name[COLOCATE USING colocation_name]|[NO]FALLBACK[PROTECTION]|WITH JOURNAL TABLE=table_specification|[NO]LOG|[NO|DUAL][BEFORE]JOURNAL|[NO|DUAL|LOCAL|NOT LOCAL]AFTER JOURNAL|CHECKSUM={DEFAULT|ON|OFF}|FREESPACE=integer[PERCENT]|mergeblockratio|datablocksize|blockcompression|isolated_loading}where:mergeblockratio:{DEFAULT MERGEBLOCKRATIO|MERGEBLOCKRATIO=integer[PERCENT]|NO MERGEBLOCKRATIO}datablocksize:DATABLOCKSIZE={data_block_size[BYTES|KBYTES|KILOBYTES]|{MINIMUM|MAXIMUM|DEFAULT}DATABLOCKSIZE}blockcompression:BLOCKCOMPRESSION={AUTOTEMP|MANUAL|ALWAYS|NEVER|DEFAULT}[,BLOCKCOMPRESSIONALGORITHM={ZLIB|ELZS_H|DEFAULT}][,BLOCKCOMPRESSIONLEVEL={value|DEFAULT}]isolated_loading:WITH[NO][CONCURRENT]ISOLATED LOADING[FOR{ALL|INSERT|NONE}]Examples:in sql materialization definition file:{{config(materialized="table",table_option="NO FALLBACK")}}{{config(materialized="table",table_option="NO FALLBACK,NO JOURNAL")}}{{config(materialized="table",table_option="NO FALLBACK,NO JOURNAL,CHECKSUM = ON,NO MERGEBLOCKRATIO,WITH CONCURRENT ISOLATED LOADING FOR ALL")}}in seed configuration:seeds:<project-name>:table_option:"NO FALLBACK"seeds:<project-name>:table_option:"NO FALLBACK, NO JOURNAL"seeds:<project-name>:table_option:"NOFALLBACK,NOJOURNAL,CHECKSUM=ON,NOMERGEBLOCKRATIO,WITHCONCURRENTISOLATEDLOADINGFORALL"For details, seeCREATE TABLE documentation.with_statistics- should statistics be copied from the base table, e.g.:{{config(materialized="table",with_statistics="true")}}This option is not available for seeds as seeds do not useCREATE TABLE ... ASsyntax.For details, seeCREATE TABLE documentation.index- defines table indices:[UNIQUE]PRIMARY INDEX[index_name](index_column_name[,...])|NO PRIMARY INDEX|PRIMARY AMP[INDEX][index_name](index_column_name[,...])|PARTITION BY{partitioning_level|(partitioning_level[,...])}|UNIQUE INDEX[index_name][(index_column_name[,...])][loading]|INDEX[index_name][ALL](index_column_name[,...])[ordering][loading][,...]where:partitioning_level:{partitioning_expression|COLUMN[[NO]AUTO COMPRESS|COLUMN[[NO]AUTO COMPRESS][ALL BUT]column_partition]}[ADD constant]ordering:ORDER BY[VALUES|HASH][(order_column_name)]loading:WITH[NO]LOAD IDENTITYe.g.:in sql materialization definition file:{{config(materialized="table",index="UNIQUE PRIMARY INDEX ( GlobalID )")}}:information_source: Note, unlike intable_option, there are no commas between index statements!{{config(materialized="table",index="PRIMARY INDEX(id)PARTITION BY RANGE_N(create_dateBETWEEN DATE '2020-01-01'AND     DATE '2021-01-01'EACH INTERVAL '1' MONTH)")}}{{config(materialized="table",index="PRIMARY INDEX(id)PARTITION BY RANGE_N(create_dateBETWEEN DATE '2020-01-01'AND     DATE '2021-01-01'EACH INTERVAL '1' MONTH)INDEX index_attrA (attrA) WITH LOAD IDENTITY")}}in seed configuration:seeds:<project-name>:index:"UNIQUEPRIMARYINDEX(GlobalID)":information_source: Note, unlike intable_option, there are no commas between index statements!seeds:<project-name>:index:"PRIMARYINDEX(id)PARTITIONBYRANGE_N(create_dateBETWEENDATE'2020-01-01'ANDDATE'2021-01-01'EACHINTERVAL'1'MONTH)"seeds:<project-name>:index:"PRIMARYINDEX(id)PARTITIONBYRANGE_N(create_dateBETWEENDATE'2020-01-01'ANDDATE'2021-01-01'EACHINTERVAL'1'MONTH)INDEXindex_attrA(attrA)WITHLOADIDENTITY"SeedsSeeds, in addition to the above materialization modifiers, have the following options:use_fastload- usefastloadwhen handlingdbt seedcommand. The option will likely speed up loading when your seed files have hundreds of thousands of rows. You can set this seed configuration option in yourproject.ymlfile, e.g.:seeds:<project-name>:+use_fastload:trueGrantsGrants are supported in dbt-teradata adapter with release version 1.2.0 and above. You can use grants to manage access to the datasets you're producing with dbt. To implement these permissions, define grants as resource configs on each model, seed, or snapshot. Define the default grants that apply to the entire project in yourdbt_project.yml, and define model-specific grants within each model's SQL or YAML file.for e.g. :
models/schema.ymlmodels:-name:model_nameconfig:grants:select:['user_a','user_b']Another e.g. for adding multiple grants:models:-name:model_nameconfig:materialized:tablegrants:select:["user_b"]insert:["user_c"]:information_source:copy_grantsis not supported in Teradata.More on Grants can be found athttps://docs.getdbt.com/reference/resource-configs/grantsCross DB macrosStarting with release 1.3, some macros were migrated fromteradata-dbt-utilsdbt package to the connector. See the table below for the macros supported from the connector.For using cross DB macros, teradata-utils as a macro namespace will not be used, as cross DB macros have been migrated from teradata-utils to Dbt-Teradata.CompatibilityMacro GroupMacro NameStatusCommentCross-database macroscurrent_timestamp:white_check_mark:custom macro providedCross-database macrosdateadd:white_check_mark:custom macro providedCross-database macrosdatediff:white_check_mark:custom macro provided, seecompatibility noteCross-database macrossplit_part:white_check_mark:custom macro providedCross-database macrosdate_trunc:white_check_mark:custom macro providedCross-database macroshash:white_check_mark:custom macro provided, seecompatibility noteCross-database macrosreplace:white_check_mark:custom macro providedCross-database macrostype_string:white_check_mark:custom macro providedCross-database macroslast_day:white_check_mark:no customization needed, seecompatibility noteCross-database macroswidth_bucket:white_check_mark:no customizationSQL generatorsgenerate_series:white_check_mark:custom macro providedSQL generatorsdate_spine:white_check_mark:no customizationexamples for cross DB macrosReplace:
{{ dbt.replace("string_text_column", "old_chars", "new_chars") }}
{{ replace('abcgef', 'g', 'd') }}Date truncate:
{{ dbt.date_trunc("date_part", "date") }}
{{ dbt.date_trunc("DD", "'2018-01-05 12:00:00'") }}datediffdatediffmacro in teradata supports difference between dates. Differece between timestamps is not supported.hashHashmacro needs anmd5function implementation. Teradata doesn't supportmd5natively. You need to install a User Defined Function (UDF):Download the md5 UDF implementation from Teradata (registration required):https://downloads.teradata.com/download/extensibility/md5-message-digest-udf.Unzip the package and go tosrcdirectory.Start upbteqand connect to your database.Create databaseGLOBAL_FUNCTIONSthat will host the UDF. You can't change the database name as it's hardcoded in the macro:CREATEDATABASEGLOBAL_FUNCTIONSASPERMANENT=60e6,SPOOL=120e6;Create the UDF. Replace<CURRENT_USER>with your current database user:GRANTCREATEFUNCTIONONGLOBAL_FUNCTIONSTO<CURRENT_USER>;DATABASEGLOBAL_FUNCTIONS;.runfile=hash_md5.btqGrant permissions to run the UDF with grant option.GRANTEXECUTEFUNCTIONONGLOBAL_FUNCTIONSTOPUBLICWITHGRANTOPTION;last_daylast_dayinteradata_utils, unlike the corresponding macro indbt_utils, doesn't supportquarterdatepart.Common Teradata-specific taskscollect statistics- when a table is created or modified significantly, there might be a need to tell Teradata to collect statistics for the optimizer. It can be done usingCOLLECT STATISTICScommand. You can perform this step using dbt'spost-hooks, e.g.:{{config(post_hook=["COLLECTSTATISTICSON{{this}}COLUMN(column_1,column_2...);"])}}SeeCollecting Statistics documentationfor more information.Support for model contractsModel contracts are supported with dbt-teradata v1.7.1 and onwards.
Constraint support and enforcement in dbt-teradataConstraint typeSupport	Platformenforcementnot_null✅ Supported✅ Enforcedprimary_key✅ Supported✅ Enforcedforeign_key✅ Supported✅ Enforcedunique✅ Supported✅ Enforcedcheck✅ Supported✅ EnforcedTo find more on model contracts please follow dbt documentationshttps://docs.getdbt.com/docs/collaborate/govern/model-contractsSupport fordbt-utilspackagedbt-utilspackage is supported throughteradata/teradata_utilsdbt package. The package provides a compatibility layer betweendbt_utilsanddbt-teradata. Seeteradata_utilspackage for install instructions.LimitationsTransaction modeBoth ANSI and TERA modes are now supported in dbt-teradata. TERA mode's support is introduced with dbt-teradata 1.7.1, it is an initial implementation.IMPORTANT NOTE: This is an initial implementation of the TERA transaction mode and may not support some use cases. We strongly advise validating all records or transformations utilizing this mode to preempt any potential anomalies or errorsCreditsThe adapter was originally created byDoug Beatty. Teradata took over the adapter in January 2022. We are grateful to Doug for founding the project and accelerating the integration of dbt + Teradata.LicenseThe adapter is published using Apache-2.0 License. Please seethe licensefor terms and conditions, such as creating derivative work and the support model.