Skip to content

Defer to prod

Defer functionality in dbt allows the user to run a subset of models or tests without having to first build their upstream parents. Usually, it leads to significant cost and time savings during testing of the dbt models. More information about this functionality is available in dbt docs.

Step 1: Enable Defer to production functionality in the "Actions" panel as below

enableDefertoprod

Step 2: Choose where your manifest file is stored for the production environment

You can either use this functionality in the local mode where your manifest files stay in your local computer. You can also use the SaaS instance to store your manifest files so multiple team members can benefit from prod or staging data.

Local Mode

Choose the location of your local manifest file as shown in the image below:

chooseLocal

SaaS Mode

To use the Altimate SaaS instance, you will need to install the altimate-datapilot python package. You can install it using the following command: pip install altimate-datapilot. This package is required to upload the manifest file to the SaaS instance. You can find more information about the package here.

First, create the dbt integration in the Altimate AI SaaS instance by going to Settings -> Integrations from left side navigation menu. This is just to create a reference where you can upload respective manifest files to point to.

If you would like to connect your on-premise storage for manifest file uploads, please contact us via chat or Slack.

addIntegration

Then, copy the command (last column in above image) from the UI to upload manifest files mapped to this integration.

The command will look something like this -

"datapilot dbt onboard --token --instance-name --dbt_core_integration_id --manifest-path 'Path/to/file'"

(You need to replace 'Path/to/file' with the path to your manifest file e.g. /Users/mrx/documents/repos/jaffle_shop/target/manifest.json)

You can use this command to run at regular intervals in your CI/CD or orchestration tool so the latest manifest is always available for reference automatically in other environments

Now, choose the right dbt integration to reference in your VSCode as below:

addIntegration

Step 3: Enable/disable favor state

Turn on favor-state if you need it. If it's turned on, the defer functionality will favor using the node defined in the referenced integration, even if the node exists in the current project.

Recorded demo video

Interactive demo

Using defer to prod with SaaS instance requires an API key. You can get it by signing up for free at www.altimate.ai Local mode doesn't require an API key.