eltPulse

Orchestration

eltPulse separates definitions (pipelines, code, assets) from orchestration (when and how work runs). You can orchestrate entirely in eltPulse — schedules, monitors, chains, slices — or export the same artifacts to Airflow, Prefect, GitHub Actions, or a custom runner.

Schedules

Schedules shows every pipeline with cron configuration. Enable sync schedules and optional separate dbt phases (run transform after load on its own cadence). Cron uses standard five-field syntax with timezone support; the cloud scheduler enqueues runs on eltPulse-managed workers unless the pipeline targets a customer gateway.

Schedule metadata is also emitted in eltpulse_workspace.yaml for external orchestrators.

Monitors (event triggers)

Monitors react to events — S3 object lands, SQS messages, custom webhooks — instead of clock time. Each monitor has its own Runs on setting:

  • eltPulse-managed — cloud evaluates checks and enqueues runs.
  • Customer gateway — your gateway evaluates S3/SQS (or custom worker) and POSTs results to the control plane.

Monitors link to a saved connection when cloud API credentials are required. Configure monitors and slice values on Orchestration in the app.

Run slices

Monitors and manual runs can pass a slice value (partition_key in generated code) to scope incremental windows or backfills. Set the partition column on Run slices; monitors can supply one value per line in partition_values. See Run slices (docs).

Pipeline chains

Pipeline chains define DAG edges between pipelines — e.g. run downstream dbt staging only after upstream SaaS sync succeeds. Native chains avoid webhook glue for in-product dependencies; use Webhooks for external systems.

Where it appears in the product

  • Pipelines — definitions, canvas, exports.
  • Schedules — cron overview.
  • Orchestration — monitors and slice-aware triggers.
  • Pipeline chains — success/failure edges between pipelines.
  • Runs — execution history and telemetry.
  • Metrics — trends, filters, alert rules.
  • eltpulse_workspace.yaml — scheduling and resilience metadata for portable automation.

Roadmap

Deeper sensor UX, cross-workspace orchestration patterns, and more native scheduling features are on the roadmap. Targeting and data model details: Concepts · gateway deployment: Gateway.