nano SIEM
Pipelines

Pipelines

Supported in:Self-hosted

Process a log source's events with ordered steps before they are parsed and stored

Use a pipeline to process a log source's events before they are stored. A pipeline is the source's parser written as one ordered list of processing steps, and it runs in Vector for every event routed to that source: push events sent with the source's token, cloud pull, and pull feeds.

Use steps to unpack raw JSON, remove heartbeat events, redact fields, enrich from a lookup snapshot, or reduce duplicate traffic. Each enabled step receives the previous step's output; disabled steps pass events through. A step can emit zero, one, or multiple events.

Pipelines are available on every deployment and both editions; only pivt proposals aren't in the open-source edition. Open them under Ingestion → Pipelines. The deployment's capabilities and your permissions determine which controls are available: draft preview runs only where the isolated preview worker is enabled (NANO_PIPE_PREVIEW_ENABLED=true), and live capture only where the live tap relay is configured. You cannot release an unavailable step.

To view a pipeline, you need log_sources:view. To edit its draft, you need log_sources:edit. To publish or roll back a release, you need log_sources:deploy. Event previews and live capture also require search:execute and access to the source.

Drafts and production

Editing, saving, and previewing a draft do not change production. Review release freezes a validated snapshot of the draft; Publish release publishes it as a new version of the log source and deploys it to Vector. It is live once Vector has loaded the new configuration. Versions are kept for rollback.

Start with Getting started, then use the Step reference. Live capture supplies incoming samples, and pivt can propose draft changes.

Troubleshoot

Need help with these checks? Contact support with the deployment, request ID and time with timezone.

A draft or release is unavailable

Check your permissions and the deployment's pipeline capabilities. Save and validate the draft before reviewing a release, then confirm the new version in release history. See pipeline setup.

What's next

On this page