Column Lineage
Lineage is available as model level lineage and column level lineage. You need to add an API key to view column level lineage. You can view model level lineage without an API key.
Model lineage
-
Different dbt entities like sources, seeds, models, tests, metrics, exposures, and model types are shown in the lineage view.
-
For applicable components, Clicking on "Details" shows a list of columns with descriptions as well as dbt tests that are written for that particular component.
Expand lineage graph
You can expand lineage at a single expansion level by clicking (+) signs on individual blocks or you can expand multiple levels of lineage by using "Expand" button as show below
Column lineage
- After clicking on "Details" for the component, you can see a list of columns. Then, you can click on a column name to change model lineage to column lineage view.
- In the column lineage view, links between components are shown as select links and non-select links. Select links are shown as solid lines if there is a direct flow of data between columns through select statements. Non-Select links are shown as dotted lines if columns appear in the condition/clause like where, join, having, etc. Check the Settings tab if you want to disable certain types of links or set specific expansion levels by default.
Code transformations
-
On each individual component, you can see small icons that show how that column was created (through transformation, just name change or passed unchanged). If code is available for a particular transformation, a small code icon is displayed.
-
When you click on the code icon, it shows the list of code transformations that were performed to create that column.
Column lineage with code transformation is also available in SaaS UI. Please refer to the section on SaaS Discovery UI
Following are a few limitations in the column level lineage
- Snapshots are not supported (coming soon)
- Operators that may result in an incomplete lineage
- Unnest
- Lateral View Flatten
- Json flatten to columns
Interactive demo - column lineage
Recorded demo - column lineage
Column lineage requires an API key. You can get it by signing up for free at www.altimate.ai