Package: dbt-prql
dbt-prqldbt-prql allows writing PRQL in dbt models. This combines the benefits of PRQL's
power & simplicitywithinqueries, with dbt's version control, lineage &
testingacrossqueries.Oncedbt-prqlin installed, dbt commands compile PRQL between{% prql %}&{% endprql %}jinja tags to SQL as part of dbt's compilation. No additional
config is required.ExamplesSimple example{%prql%}fromemployeesfilter(age|in20..30){%endprql%}...would appear to dbt as:SELECTemployees.*FROMemployeesWHEREageBETWEEN20AND30Less simple example{%prql%}fromin_process={{source('salesforce','in_process')}}deriveexpected_sales=probability*valuejoin{{ref('team','team_sales')}}[name]groupname(aggregate(sumexpected_sales)){%endprql%}...would appear to dbt as:SELECTname,sum(in_process.probability*in_process.value)ASexpected_salesFROM{{source('salesforce','in_process')}}ASin_processJOIN{{ref('team','team_sales')}}USING(name)GROUPBYname...and then dbt will compile thesourceandrefs to a full SQL query.Replacing macrosdbt's use of macros has saved many of us many lines of code, and even saved some
people some time. But imperatively programming text generation with code likeif not loop.lastis not our highest calling. It's the "necessary" part rather
than beautiful part of dbt.Here's the canonical example of macros in thedbt
documentation:{%-setpayment_methods=["bank_transfer","credit_card","gift_card"]-%}selectorder_id,{%-forpayment_methodinpayment_methods%}sum(casewhenpayment_method='{{payment_method}}'thenamountend)as{{payment_method}}_amount{%-ifnotloop.last%},{%endif-%}{%endfor%}from{{ref('raw_payments')}}groupby1Here's that model using PRQL[^1], including the prql jinja tags.{%prql%}funcfilter_amountmethod->s"sum(case when payment_method = '{method}' then amount end) as {method}_amount"from{{ref('raw_payments')}}grouporder_id(aggregate[filter_amountbank_transfer,filter_amountcredit_card,filter_amountgift_card,]){%endprql%}As well the query being simpler in its final form, writing in PRQL also gives us
live feedback around any errors, on every keystroke. Though there's much more to
come, check out the current version onPRQL
Playground.[^1]: Note that whenhttps://github.com/prql/prql/issues/82is implemented, we
can dispense with the s-string, and optionally dispense with the function.```elm
from {{ ref('raw_payments') }}
group order_id (
  aggregate [
    bank_transfer_amount = amount | filter payment_method == 'bank'        | sum,
    credit_card_amount = amount   | filter payment_method == 'credit_card' | sum,
    gift_amount = amount          | filter payment_method == 'gift_card'   | sum,
  ]
)
```

or

```elm
func filter_amount method -> amount | filter payment_method == method | sum

from {{ ref('raw_payments') }}
group order_id (
  aggregate [
    bank_transfer_amount = filter_amount 'bank'
    credit_card_amount   = filter_amount 'credit_card'
    gift_amount          = filter_amount 'gift_card'
  ]
)
```What it doesWhen dbt compiles models to SQL queries:Any text in a dbt model between{% prql %}and{% endprql %}tags is
compiled from PRQL to SQL before being passed to dbt.The PRQL complier passes text that's containing{{&}}through to dbt
without modification, which allows us to embed jinja expressions in PRQL.
(This was added to PRQL specifically for this use-case.)dbt will then compile the resulting model into its final form of raw SQL, and
dispatch it to the database, as per usual.There's no config needed in the dbt project; this works automatically on any dbt
command (e.g.dbt run) assumingdbt-prqlis installed.Installationpipinstalldbt-prqlCurrent stateCurrently this is new, but fairly feature-complete. It's enthusiastically
supported — if there are any problems, please open an issue.How does it work?It's some dark magic, unfortunately.dbt doesn't allow adding behavior beyond the database adapters (e.g.dbt-bigquery) or jinja-only plugins (e.g.dbt-utils). So this library hacks
the python import system to monkeypatch dbt's jinja environment with an
additional jinja extension on python's startup[^2].[^2]: Thanks tomtkennerly/poetry-dynamic-versioningfor the technique.This approach was discussed with the dbt teamhereandhere.This isn't stable between dbt versions, since it relies on internal dbt APIs.
The technique is also normatively bad — it runs a few lines of code every time
the python interpreter starts — whose errors could lead to very confusing bugs
beyond the domain of the problem (though in the case of this library, it's small
and well-constructed™).If there's ever any concern that the library might be causing a problem, just
set an environment variableDBT_PRQL_DISABLE=1, and this library won't
monkeypatch anything. It's also fully uninstallable withpip uninstall dbt-prql.RoadmapOpen to ideas; at the moment it's fairly feature-complete. If we were
unconstrained in dbt functionality:If dbt allowed for external plugins, we'd enthusiastically move to that.We'd love to have this work on.prqlfiles without the{% prql %}tags;
but with the current approach that would require quite invasive
monkeypatching.If we could add the dialect in automatically (i.e.prql dialect:snowflake),
that would save a line per model.If we could upstream this into dbt-core, that would be awesome. It may be on
PRQL to demonstrate its staying power before that, though.We may move this library to thehttps://github.com/prql/PyPrqlorhttps://github.com/prql/prqlrepos. We'd prefer to keep it as its own package
given the hackery above, but there's no need for it to be its own repo.