Skip to content

How Braidplane works

For an assessment of an exact Git change before applying it, see Review a pipeline change with Proof.

Imagine an operations team that needs to combine order records with customer information and send a useful result to another system. Braidplane lets the team describe that work as a pipeline, select where it runs, and inspect each execution.

This is a reading tour. You do not need to install anything or connect a system to follow it.

1. Choose the systems involved

A connector describes how Braidplane talks to a kind of system, such as a database, HTTP API, CRM, or file store. A data source is one configured connection to a specific system. Despite the name, a data source can also be a destination when its connector and permissions allow writes.

For our example, identify where orders and customers live and where the result should go. These choices determine which connectors and operations to use; PostgreSQL and S3 are not platform requirements.

Explore the connector capabilities before choosing systems when you are ready to build a workflow.

2. Describe the work

A pipeline connects explicit steps:

Read orders ─────┐
                 ├─ Join and select fields ─ Send the result
Read customers ──┘

A source operation specifies which records or files to read. A transform specifies how to combine or reshape compatible inputs. A sink operation specifies the write to perform. For example, an SQL transform can join row-shaped order and customer inputs by customer ID.

Data contracts define the shapes expected between steps. Choose operations that produce the input your next step needs: file bytes, JSON documents or rows.

Pipeline definitions are versioned through Git. Credentials belong to the configured connections, outside the pipeline definition.

3. Organize the workflow

An organization owns the connections and controls who can access them. A project groups related pipelines. A project environment binds the pipeline's logical connection names to saved data sources.

For example, the pipeline can refer to orders while a development environment maps that name to a connection containing synthetic orders. Another environment can have its own binding. The definition can stay the same while the selected connections differ.

4. Validate and deploy a version

Validation checks the pipeline structure, bindings and cached metadata before execution. Refresh schemas separately when a connected system changes.

A deployment selects an immutable pipeline version for a project environment. This makes the version and connection bindings explicit before execution.

5. Run and inspect

A run executes the deployed workflow. Inspect its status, step timeline, and failure diagnostics to understand what completed and where execution stopped. If a step fails, inspect its diagnostics before deciding whether to retry; write and replay behavior depends on the connector and operation.

For our example, inspect the reads, join and write in the run timeline, then compare the destination records with the expected result.

Where to go next

The PostgreSQL-to-S3 walkthrough includes sample data and a complete pipeline for teams with a database, bucket and Git repository.