Git workflow for dbt with Amazon Redshift

Source

yaml

id: dbt-redshift
namespace: company.team

tasks:
  - id: git
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: clone_repository
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/dbt-example
        branch: main

      - id: dbt
        type: io.kestra.plugin.dbt.cli.DbtCLI
        taskRunner:
          type: io.kestra.plugin.scripts.runner.docker.Docker
        containerImage: ghcr.io/kestra-io/dbt-redshift:latest
        profiles: |
          my_dbt_project:
            outputs:
              dev:
                type: redshift
                host: myhostname.us-east-1.redshift.amazonaws.com
                user: "{{ secret('REDSHIFT_USER') }}"
                password: "{{ secret('REDSHIFT_PASSWORD') }}"
                port: 5439
                dbname: analytics
                schema: dbt
                autocommit: true # autocommit after each statement
                threads: 8
                connect_timeout: 10
            target: dev
        commands:
          - dbt deps
          - dbt build

About this blueprint

Data

This flow:

clones a dbt Git repository from GitHub
pulls a public container image with the required dependencies
runs dbt CLI commands within a container

This flow assumes that your Redshift credentials are stored as secrets.

To run a flow from this blueprint, you can simply copy-paste the contents of your ~/.dbt/profiles.yml into the profiles section and you're good to go!

Working Directory

Clone

Dbt CLI

Docker

More Related Blueprints

Data

Airbyte Cloud ingestion with dbt Cloud transformation

Data

Trigger multiple Airbyte Cloud syncs in parallel, then run a dbt job

Data

Trigger a single Airbyte Cloud sync on schedule

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra