eltPulse

dbt transformations

eltPulse supports dbt in two ways: first-class workspace projects (like Snowflake dbt Projects or dbt Cloud) and in-pipeline post-load runs after connector sync completes.

Workspace dbt projects

Register Git-backed or local-path projects at /catalog/dbt. Each DbtProject row stores package path, Git URL/branch, target schema, run scope, schedule, and optional warehouse connection. You can:

  • Run dbt standalone (runs without a pipeline id).
  • Link one project to one or more pipelines for shared EL+T config.
  • Scaffold from the Transform hub or connector staging packages.

Enable in the builder or canvas

  1. Open a pipeline in the form builder or canvas (connector sync pipelines support in-pipeline dbt).
  2. Under post-transform (builder) or the transform node inspector (canvas), choose dbt.
  3. Use the workspace dbt project picker to link a registered project, or configure inline (package_path, schema, branch, selector).
  4. Save — the pipeline stores dbtProjectId when linked; config merges into sourceConfiguration.dbt for codegen.

AI Builder

The AI Builder can generate EL+T pipelines with post_transform_type=dbt, list existing workspace projects, and set dbtProjectId on the save payload. See AI Builder docs.

Configuration shape

Transform settings persist on the pipeline under sourceConfiguration.dbt (legacy key: dlt_dbt):

{
  "enabled": true,
  "project_id": "clx…",
  "package_path": "https://github.com/org/dbt-analytics.git",
  "dataset_name": "analytics",
  "repository_branch": "main",
  "run_scope": "all",
  "selector": "tag:nightly",
  "slice_value_var": "elt_partition_value",
  "slice_column_var": "elt_partition_column"
}

Partitioned / sliced runs

When you run with a partition key, eltPulse passes slice context into dbt as vars. Default names are elt_partition_value and elt_partition_column — override with slice_value_var and slice_column_var to match existing models that call var('my_slice').

Connector staging packages

Many verified connectors have community staging packages on the dbt Hub (e.g. stripe_source). See the transform hub to browse by connector, or start from the connector catalog.

Permissions

Creating, editing, linking, and running dbt projects requires member (pipeline write). Catalog editors and viewers may browse project detail pages but cannot save or trigger runs. See Catalog & RBAC.

What we don't do

  • Host dbt Cloud, the dbt IDE, or semantic-layer authoring.
  • Auto-generate a full mart layer for every connector — you bring or add dbt packages.
  • Run dbt inside database-replication-only pipelines — use a connector sync pipeline or run dbt separately in CI for pure replication workloads.

dbt projects (app) · Pipelines · AI Builder