Performance & Benchmarks

This page reports measured search performance for real incident-response and threat-hunting workloads — not synthetic micro-benchmarks. Every number below was captured against a live nano environment running production conditions, and every latency is the search engine's own server-side execution time unless noted.

TL;DR

Real, source-scoped investigations are fast at production scale:

• Triage queries (minutes to 1h) return in tens of milliseconds.
• Hunts (6–24h, multi-filter) land in the double-to-low-hundreds of milliseconds.
• Advanced, multi-stage IR (24h–7d) generally stays under ~2 seconds — even while the box ingests ~200 EPS and runs 26 detections.

The only slow queries are deliberate worst-case probes — bare, unscoped keyword scans over a full day — which are full-table scans no analyst actually runs (see the appendix). The lever that keeps everything else fast is source-scoping, not small time windows.

Real-world load test

The corpus is 94 validated queries across 14 SOC/IR domains and 46 distinct MITRE ATT&CK techniques, replayed at each query's natural analyst time window. The load test ran 3 concurrent analysts for 3 minutes against the single replica, on top of the live ingest + detection load.

1,117 queries executed · 0.18% error rate · every analyst-tier SLO met.

By SOC domain

Server-side latency, broken down by investigation domain (most scope to a single source_type; cross-source IR and impossible-travel intentionally span several):

Endpoint/EDR is the one heavy tail: two of its queries match on Sysmon fields that aren't promoted to indexed UDM columns (they live in the raw message), so they fall back to a full-text scan. Everything else stays sub-second-to-~1.5 s at p95.

Representative queries

These are real nPL queries from the corpus, with their measured single-shot server-side latency against the live demo. They're copy-pasteable starting points for your own hunts.

Triage — scoped, minutes to 1 hour

Encoded PowerShell · 1h · 40 ms

source_type=windows_sysmon action=process_create command_line=/EncodedCommand/i
| stats count by src_host, user

Brute force by account · 1h · 33 ms

source_type=windows_event action=logon_failure
| stats count as failures, dc(src_ip) as src_ips by user
| where failures > 20

Blocked egress · 1h · 24 ms

source_type=conduit_proxy action=deny OR action=blocked
| stats count by src_ip, dest_host

Failed AssumeRole · 1h · 43 ms

source_type=aws_cloudtrail action=assumerole status=failure
| stats count by user, src_ip

Web auth abuse · 1h · 20 ms

source_type=apache_access (http_status_code=401 OR http_status_code=403)
| stats count by src_ip, uri_path

Hunt — multi-filter, 6 to 24 hours

LOLBin to external C2 · 6h · 93 ms

source_type=windows_sysmon action=network_connection (process_name=powershell.exe OR process_name=rundll32.exe)
| stats count by process_name, dest_ip, src_host

Failed-then-succeeded auth · 6h · 79 ms

source_type=windows_event
| stats count(eval(action="logon_failure")) as fails, count(eval(action="logon_success")) as wins by user
| where fails > 30 AND wins > 0

Exfil by volume · 6h · 48 ms

source_type=conduit_proxy action=allow
| stats sum(bytes_out) as bytes_out by src_ip, dest_host
| sort -bytes_out

DNS-based C2 · 24h · 131 ms

source_type=windows_sysmon action=dns_query
| stats count as queries, dc(dest_host) as uniq_domains by src_host

Advanced / retro IR — parameter-heavy, multi-stage, 24h to 7d

Compromise then escalate · 24h · 62 ms

source_type=windows_event
| stats count(eval(action="logon_failure")) as fails, count(eval(action="special_privileges_assigned")) as priv_grants by user
| where fails > 100 AND priv_grants > 100

Single-host timeline · 6h · 36 ms

src_host="ws-exec-087.corp.local"
| timechart span=1h count by action

AccessDenied spike · 24h · 269 ms

source_type=aws_cloudtrail status=failure
| stats count as denials, dc(action) as uniq_actions by user

Concurrency & scaling

To find the saturation point, we ramped concurrent virtual analysts 5 → 10 → 25 → 50 → 100 against a single replica, reached directly via port-forward — deliberately bypassing both the load balancer and the search admission controller. This is a worst-case "one pod, no guardrails" probe.

The important read: server-side query execution stays flat at ~250 ms p50 across the entire ramp. ClickHouse is not the bottleneck. What saturates past ~10 concurrent heavy analysts is a single replica's request admission — and note the round-trip p95 actually drops at high concurrency (930 ms → 302 ms) because shed requests fail fast rather than queuing. The error rate, not latency, is what moves.

Appendix: worst-case stress reference

For completeness, here's the ceiling — deliberately pathological queries that no real investigation runs: bare, unscoped keyword tokens and dense-column regex, which force a full-table scan with nothing to prune on.

This cost is window-bound, not match-bound: error | stats count is a flat ~3 s over 24h regardless of how many rows match (even a token with zero matches takes ~3 s), and drops to 0.21 s at a 1h window. The bare-keyword 24h path is the only thing that full-scans.

Methodology

• Driver: k6, replaying a 94-query corpus (benchmarks/k6/search_realistic.js) and a discrete-stage concurrency ramp (benchmarks/k6/search_concurrency_ramp.js).
• Auth: read-only JWT via /api/auth/login; requests hit POST /api/search with table_view and skip_field_stats set (the same shape the Search UI sends).
• Server-side timing is the engine's reported execution_time_ms (ClickHouse execution), captured separately from client round-trip so transport limits are distinguishable from engine limits.
• Conditions: live tenant under ~200 EPS ingestion with 26 detection rules running — not a quiesced box.