Direct OCSF Ingestion
Send events that Tenzir, Cribl or a custom forwarder has already shaped to OCSF, through a parser or straight into ClickHouse
Send events that your Tenzir, Cribl, or custom pipeline has already shaped to OCSF. nano stores every event in one version-agnostic ClickHouse table, nanosiem.ocsf_logs; see OCSF schema.
You have two paths:
| Through ingest, with a parser | Direct write into ClickHouse | |
|---|---|---|
| Who validates the event | A parser for the source | Your pipeline |
| Transport | HTTP, HEC or OTLP with an ingest token | ClickHouse native protocol (:9000) or HTTP (:8123) |
| Reaches | Every deployment | Deployments on the OCSF schema profile whose ClickHouse your pipeline can reach on a trusted network |
| Bypasses Vector | No | Yes: no parser, no pipeline steps |
| Best for | Cribl, custom forwarders, any sender over the internet | Tenzir on the same network as nano |
Send pre-shaped events through ingest
- Create a log source for the feed, for example
my_firewall, and issue an ingest token bound to it. The source becomesmetadata.log_sourceon every stored event. - Deploy a parser that claims that source in its
match_values. For events your pipeline has already shaped, the parser can be small: parse the JSON, check it's what you expect, setraw_dataand any missingmetadatafields, and assign it to the root. See Parser extensions for the parser contract. - Send with the protocol your pipeline speaks:
- Tenzir or a custom forwarder: HTTP (
POST /ingest) or HEC (/services/collector/event). - Cribl Stream: its HEC destination. See Cribl Stream.
- Endpoints and auth: Ingest data and HEC.
- Tenzir or a custom forwarder: HTTP (
Example with an ingest token bound to my_firewall (the token decides the source):
curl -sS --fail-with-body -X POST "https://<ingest-host>/ingest" \
-H "Authorization: Bearer $NANO_TOKEN" \
-H "Content-Type: application/json" \
-d '{"class_uid": 4001, "category_uid": 4, "activity_id": 6, "type_uid": 400106, "severity_id": 1, "time": 1781234567890, "src_endpoint": {"ip": "203.0.113.7"}, "dst_endpoint": {"ip": "198.51.100.9", "port": 443}, "metadata": {"product": {"name": "my-fw"}, "version": "1.9.0"}}'On open core, send to http(s)://<host>/ingest.
Then check it landed:
metadata.log_source="my_firewall" | stats count by class_uid, src_endpoint.ipWrite directly into ClickHouse
On the OCSF schema profile (NANO_SCHEMA_PROFILE=ocsf, the default for new installs), a pipeline can insert OCSF rows straight into ClickHouse, bypassing Vector:
| Value | |
|---|---|
| Table | nanosiem.ocsf_logs_raw. Tenzir 6.4–6.5 writes the legacy entrypoint nanosiem.ocsf_logs_native_raw instead |
| Row shape | {"event": <OCSF object>, "source_type": "<source>"}; leave out timestamp and id, which ClickHouse derives |
| Native protocol | :9000, for example Tenzir's to_clickhouse operator (Tenzir 6.6 or later) with tls=false, mode="append" |
| HTTP | :8123, INSERT … FORMAT JSONEachRow |
| User | nanosiem_ingest, which can only INSERT. Its password is CLICKHOUSE_INGEST_PASSWORD in .env (Compose) or clickhouse_ingest_password in the chart secret (Kubernetes) |
Authenticate as nanosiem_ingest, never the application user, and never expose ClickHouse ports to the internet. ClickHouse is on the internal Docker network in the Compose stack; publish it only to a trusted network your pipeline runs on.
The Tenzir sink, copied verbatim:
unordered { batch 65536, timeout=2s }
to_clickhouse table="nanosiem.ocsf_logs_raw",
host=env("NANO_CH_HOST").otherwise("clickhouse"),
port=int(env("NANO_CH_NATIVE_PORT").otherwise("9000")),
user=env("NANO_CH_INGEST_USER").otherwise("nanosiem_ingest"),
password=env("CLICKHOUSE_INGEST_PASSWORD"),
tls=false, mode="append"Batch before the sink: real OCSF is schema-heterogeneous, and without unordered { batch } every event becomes a one-row INSERT that can exhaust the Tenzir node's memory under load.
Bundled Tenzir node
The installer's ocsf+tenzir schema profile (NANO_SCHEMA_PROFILE=ocsf+tenzir) also runs a Tenzir node, nano-tenzir, with this sink pre-wired. It accepts OCSF NDJSON on http://<host>:9095/ without authentication and binds to 127.0.0.1 unless you set NANO_TENZIR_BIND_ADDRESS. Add your sources and OCSF mapping to tenzir/pipeline.tql in the install directory, then restart the tenzir service.
OCSF shape rules that matter
| Rule | Why |
|---|---|
Set metadata.version to the OCSF version you emit | Every row keeps its own version; field resolution uses the matching contract |
class_uid and time are required; time in epoch milliseconds | timestamp is derived from time |
type_uid = class_uid × 100 + activity_id | Checked by the parser validator |
Put the untouched original line in raw_data (a string) | Kept for review; keyword search covers it |
Fields with no OCSF home go under unmapped | Keeps the OCSF object clean and searchable |
Lowercase source_type and values nano lowercases at ingest (IPs, hostnames, hashes) | Equality searches compare those columns as stored |
Hashes are arrays of {algorithm_id, value} | Searched as file.hashes.value |
Guard required columns against null (for example source_type.otherwise("unknown")) | A null in a required column drops the row on a direct write |
The field paths nano searches are in the OCSF field reference.
Troubleshoot
Need help with these checks? Contact support with the deployment, request ID and time with timezone.
Events do not have the expected OCSF fields
Through ingest, check that the deployed parser claims the source and emits the full OCSF object at the root. On a direct write, check the row shape and the shape rules above: there's no parser to fix it.
Tenzir fails with wrong version number
to_clickhouse defaults to TLS; the native port on the internal network is plaintext. Set tls=false.
What's next
- OCSF schema: how nano stores events
- Supported sources: the bundled OCSF parsers
- Push vs pull routing: how nano picks a parser