streamstats

Calculate running statistics per event without aggregating rows. Add cumulative metrics and window-based calculations.

Description

The streamstats command computes statistics across events while preserving all rows. Unlike stats which aggregates data, streamstats adds new fields to each event containing running totals, moving averages, or previous values.

This is essential for detecting sequences, calculating deltas, and identifying anomalies based on historical context.

Syntax

... | streamstats [current=<bool>] [window=<int>] <function>([field]) [as <alias>] [by <field>]

Optional Arguments

current
Syntax: current=true|false
Description: Include current row in calculation
Default: true

window
Syntax: window=<int>
Description: Number of preceding rows to include (sliding window)
Default: All preceding rows

by
Syntax: by <field>
Description: Partition calculations by field (separate running stats per group)

Examples

Running event count

* | streamstats count() as event_num

Previous timestamp

* | streamstats current=false last(timestamp) as prev_ts by dest_host
  | eval time_diff = timestamp - prev_ts

Rolling average (10 events)

* | streamstats window=10 avg(bytes) as rolling_avg by src_ip

Cumulative sum

* | streamstats sum(bytes) as total_bytes by user

Detect rapid succession

action=login
| sort timestamp
| streamstats current=false last(timestamp) as prev_login by user
| eval seconds_since_last = timestamp - prev_login
| where seconds_since_last < 5

Moving average for anomaly detection

* | streamstats window=100 avg(response_time) as avg_time, stdev(response_time) as std_time
  | eval is_anomaly = response_time > (avg_time + (3 * std_time))
  | where is_anomaly=true

Cumulative failed logins

action=login
| sort timestamp
| streamstats sum(eval(status="failure")) as cumulative_failures by src_ip
| where cumulative_failures > 10

Previous value comparison

* | sort timestamp
  | streamstats current=false last(status) as prev_status by endpoint
  | where status != prev_status

Running distinct count

* | streamstats dc(dest_port) as unique_ports by src_ip
  | where unique_ports > 50

Session numbering

* | sort timestamp
  | streamstats count() as session_num by user

Usage Notes

Order matters: Use sort before streamstats to ensure correct sequence.

Memory: Window-based calculations are memory-efficient. Full running stats require tracking all previous values.

Partitioning: by clause creates separate running stats for each group.

Current row: Set current=false to exclude current row (useful for "previous value" calculations).

Related Commands

• stats - Aggregate data (reduces rows)
• eventstats - Add stats to all rows (no running calculation)
• sort - Order events before streamstats