Direct OCSF Ingestion

If you already run a data aggregator that parses and shapes events to OCSF — Tenzir, Cribl, or a custom forwarder — you can write those records directly into nano's ClickHouse, bypassing nano's Vector pipeline entirely. Your aggregator owns parsing and normalization; nano owns storage, enrichment, search, and detection.

Already using an aggregator? Pick who owns normalization

If you already run Tenzir, Cribl, or a similar tool, there are two ways to feed nano — and the choice is really about who parses and normalizes the data: nano, or your aggregator.

Most teams should start by pointing at Vector (left column): ship raw events with a known source_type and let nano handle parsing, enrichment, storage, and detection. No ClickHouse credential, no schema commitment, works on either profile. See Splunk HEC (Cribl and Splunk forwarders speak this natively) or the push/pull routing model.

Choose direct ingestion (this page) when your aggregator is already your OCSF normalization layer and you want nano purely as the analytics + detection backend — no nano-side parsing. nano ships a SIEM-only deployment for exactly this: the full stack minus Vector. One tradeoff to weigh up front: nano keeps the promoted OCSF columns, message, and an unmapped spill — not a verbatim copy of your original event blob (see How it works).

docker compose -f docker-compose.siem-only.yml up -d

How it works

You INSERT OCSF records into the nanosiem.ocsf_logs_raw landing table (a Null-engine table — nothing is stored there). An insert-time materialized view derives every promoted, enriched, and prevalence column into nanosiem.ocsf_logs from the event JSON, so a third-party INSERT gets exactly the same treatment as a Vector-lane row:

• Promoted columns materialize from the event (dotted OCSF paths: src_endpoint.ip, actor.process.cmd_line, …) — indexed and searchable immediately.
• Geo / ASN enrichment is applied at insert when the event isn't already enriched (pre-enriched OCSF passes through untouched).
• Prevalence — direct rows both feed and read the shared prevalence universe.
• Aggregation surfaces — entity first/last-seen, identity observations, per-source telemetry, and NAT detection all populate from direct-written rows.
• Row identity — a server-minted UUIDv7 id for the event inspector.

The credential

Authenticate as the dedicated INSERT-only ClickHouse user — never the application user.

This credential cannot read log content, query other tables, or write to the enrichment/prevalence source tables. Never hand a foreign pipeline the shared nanosiem application user — it can read and write the entire database.

Wire format

INSERT rows with JSONEachRow over the ClickHouse HTTP interface (port 8123).

curl "http://<clickhouse-host>:8123/" \
  --user "nanosiem_ingest:$CLICKHOUSE_INGEST_PASSWORD" \
  --data-binary 'INSERT INTO nanosiem.ocsf_logs_raw (event, source_type) FORMAT JSONEachRow
{"event": {"class_uid": 4001, "category_uid": 4, "type_uid": 400101, "activity_id": 1, "severity_id": 1, "time": 1781234567890, "src_endpoint": {"ip": "203.0.113.7"}, "dst_endpoint": {"ip": "198.51.100.9", "port": 443}, "metadata": {"product": {"name": "my-fw"}, "version": "1.8.0"}}, "source_type": "my_firewall"}'

Required per row

Optional: timestamp (derived from event.time when omitted — prefer omitting it), id (server-minted UUIDv7 when omitted).

time fallback semantics

The timestamp column derives from event.time and never silently loses a row:

Only the first row is OCSF-compliant: timestamp_t is defined as epoch milliseconds. Epoch seconds, microseconds, and RFC3339 strings are nano compatibility extensions, not part of the spec — lean on them for a quick start, but emit epoch ms if you want events that are portable to other OCSF consumers.

Lowercase rule

Keep source_type — and any other categorical routing identifier you set on the row — lowercase. The server cannot rewrite client-provided values; the query layer tolerates mixed-case source_type, but lowercasing keeps facets, group-bys, and dashboards from splitting one feed into multiple values. This applies to categorical labels, not to data: fields inside the event object (IP addresses, hostnames, and the like) are stored exactly as your aggregator emits them and don't need lowercasing.

Async inserts

The server enables async_insert with wait_for_async_insert=1, so HTTP 200 is a durability ACK — the response waits for the flush, and a failing batch returns a retryable error. Batch your events (don't send one per request); expect up to ~1s of flush latency, which batching absorbs. Retrying is safe: a block-level dedup window means re-sending a byte-identical batch is acknowledged but writes nothing.

Worked example: Tenzir

A complete Tenzir pipeline that maps events to OCSF and writes them in via the native to_clickhouse operator. Requires Tenzir ≥ 6.4.0. Validated end-to-end against Tenzir v6.4.0.

// Shape your events as OCSF (or read records that already are), then:
from_file "/data/events.ndjson" { read_ndjson }
this = { event: this, source_type: "my_tenzir_feed" }
to_clickhouse table="nanosiem.ocsf_logs_native_raw",
  host="<clickhouse-host>", port=9000,
  user="nanosiem_ingest", password=secret("CLICKHOUSE_INGEST_PASSWORD"),
  tls=false, mode="append"

A few notes:

• Keep the password in a Tenzir secret. password=secret("CLICKHOUSE_INGEST_PASSWORD") resolves the value at run time from the node config, an environment variable, or the connected Tenzir platform — the credential never lands in the persisted pipeline definition. Don't inline the literal.
• Validate before you send (recommended). Running Tenzir's ocsf::cast conforms your record to the official OCSF schema — it type-checks fields, enforces required attributes, and reorders into canonical form — a solid gate before the sink. One caveat: ocsf::cast serializes time-typed fields as RFC3339 strings; nano's time fallback parses them, but keep time as an integer (epoch ms) where you can.
• Wrap each OCSF record in the wire shape {event: <ocsf record>, source_type: "<your feed>"} as the last transform before the sink.
• Target ocsf_logs_native_raw, not ocsf_logs_raw. to_clickhouse's append mode validates every column type on the target and rejects ocsf_logs_raw's LowCardinality(String) / timezone-qualified DateTime64(3, 'UTC') columns. ocsf_logs_native_raw exposes only event JSON + source_type String and forwards into the identical derivation chain. to_clickhouse also defaults to TLS — pass tls=false for the plaintext native port.

A self-contained smoke test with a literal event:

from {
  event: {
    class_uid: 4001,
    category_uid: 4,
    type_uid: 400101,
    activity_id: 1,
    severity_id: 1,
    time: 1781234567890,
    src_endpoint: {ip: "10.0.0.50", port: 51234},
    dst_endpoint: {ip: "8.8.8.8", port: 443},
    metadata: {product: {name: "tenzir"}, version: "1.8.0"}
  },
  source_type: "my_tenzir_feed"
}
to_clickhouse table="nanosiem.ocsf_logs_native_raw",
  host="<clickhouse-host>", port=9000,
  user="nanosiem_ingest", password=secret("CLICKHOUSE_INGEST_PASSWORD"),
  tls=false, mode="append"

Worked example: Cribl

In Cribl Stream, send shaped OCSF events to a ClickHouse destination (or a generic HTTP destination) configured for the HTTP interface:

• URL: http://<clickhouse-host>:8123/?query=INSERT INTO nanosiem.ocsf_logs_raw (event, source_type) FORMAT JSONEachRow&wait_for_async_insert=1
• Method: POST, format JSONEachRow (one JSON object per line)
• Auth: HTTP Basic, nanosiem_ingest + CLICKHOUSE_INGEST_PASSWORD
• Payload: each record is {"event": { …OCSF… }, "source_type": "<your feed>"}

Use a Cribl Eval/Serialize function to nest your shaped OCSF object under event and set a lowercase source_type before the destination.

Verify it landed

The nanosiem_ingest credential is INSERT-only — it can't run a SELECT (see The credential), so verify from the nano search bar rather than querying ClickHouse as the ingest user. After sending a batch, the data is queryable natively:

source_type=my_tenzir_feed | stats count by src_endpoint.ip

Promoted columns, insert-time geo/ASN enrichment, prevalence, entity pages, and detection-rule matches all work on direct-written rows exactly as they do for Vector-collected data.

To confirm straight against ClickHouse instead, run the count with a read-capable credential — the nanosiem application user — never the INSERT-only nanosiem_ingest:

curl -s "http://<clickhouse-host>:8123/" \
  --user "nanosiem:<app-password>" \
  --data-binary "SELECT source_type, count() FROM nanosiem.ocsf_logs
    WHERE _inserted_at > now() - INTERVAL 10 MINUTE GROUP BY source_type"

Troubleshooting

Known limitations

• Self-managed deployments that add materialized views reading new columns must extend the nanosiem_ingest grants in lockstep (the managed operator upgrade does this automatically).
• A flush rejected by a missing grant can return an error after the ocsf_logs parts are committed — the rows are stored even though the INSERT reported failure. The dedup window makes a byte-identical retry safe.