nano SIEM
Search Commands

prevalence

Supported in:Self-hosted

Filter or annotate results by how many of your hosts have seen a hash, domain or IP: find the rare and the brand new.

Use prevalence to filter or annotate your results by how common a file hash, domain, or public IP is across your environment. You can use rare or newly seen artifacts as investigation leads.

Prevalence is per deployment: counts come only from your own events.

Syntax

... | prevalence <condition> [[AND] <condition> ...] [window=24h|7d|30d] [enrich=true]
... | prevalence enrich=true [window=24h|7d|30d]
... | prevalence <field>[, <field> ...] [window=24h|7d|30d]

Filter form

Keep only rows whose artifacts satisfy every condition.

Condition fieldComparesValue
hash_prevalenceNumber of hosts that have seen the row's file or process hashan integer
domain_prevalenceNumber of hosts that have contacted the row's domainan integer
hash_first_seenWhen the hash was first seen in your environmentnow() - <n>[s|m|h|d], or now() - INTERVAL <n> <UNIT>
domain_first_seenWhen the domain was first seensame as above

Operators: <, <=, >, >=, =, !=. A bare number after now() - is seconds.

hash_first_seen > now() - 24h means "first seen within the last 24 hours".

Enrich form

prevalence enrich=true keeps every row and adds prevalence columns, including:

ColumnMeaning
host_countHosts that have seen the artifact (the lowest count when the row has several artifacts)
is_raretrue when the artifact is at or below your rarity threshold
total_occurrencesTotal events that carried the artifact
first_seen, last_seenFirst and last time the artifact was seen
prevalence_score0–100 prevalence score
prevalence_artifactThe artifact value(s) checked, comma-separated
prevalence_typeThe artifact type(s): domain, ip, hash
prevalence_minSame as host_count
prevalence_first_seen, prevalence_last_seenSame as first_seen, last_seen

The positional form prevalence <field>[, <field>] enriches the values of the named fields only, for example prevalence http_request.url.hostname. It adds the same columns and does not filter.

Options

OptionDefaultDescription
window30dLook-back for the prevalence counts: 24h, 7d or 30d. interval= is an alias. window=1h is rejected: "prevalence window=1h is not supported; use 24h, 7d, or 30d".
enrichfalse (filter form)With the filter form, also add the enrich columns to the rows that pass.
threshold=—Accepted on the positional form and ignored.

Private IP addresses are not counted as artifacts.

Examples

Process executions whose hash three or fewer hosts have run in the last week:

class_uid=1007 | prevalence hash_prevalence <= 3 window=7d | table time, device.hostname, actor.user.name, process.name, process.file.hashes.value

Hashes first seen in your environment within the last 24 hours:

class_uid=1007 | prevalence hash_first_seen > now() - 24h | stats count by process.name, process.file.hashes.value

Web traffic to rare domains that are also new this week:

class_uid=4002 | prevalence domain_prevalence < 5 AND domain_first_seen > now() - 7d | stats count by http_request.url.hostname, src_endpoint.ip

Annotate proxy traffic and keep the rare destinations:

metadata.log_source=conduit_proxy | prevalence enrich=true | where is_rare=true | stats count by http_request.url.hostname | sort -count

Annotate one field only:

class_uid=4002 | prevalence http_request.url.hostname | table time, src_endpoint.ip, http_request.url.hostname, host_count, first_seen

Prevalence stamped at ingest

Events also carry the prevalence of their artifacts as of ingest in the OCSF enrichments array: entries with name prevalence, provider nano and type reputation. short_desc is new artifact, rare artifact or observed artifact, and data holds host_count, is_rare, is_new, first_seen and the other counts. These are normal fields, so you can filter on them in the base search without the prevalence command:

metadata.log_source=conduit_proxy enrichments.short_desc="new artifact" | stats count by http_request.url.hostname

The ingest stamp answers "was this rare when it happened?" and never changes. | prevalence answers "how rare is it now?". See OCSF schema for the enrichments array.

Run commands after enrichment

prevalence is never compiled into the ClickHouse query. The search splits at the first lookup, prevalence, inputlookup, resolve_identity, ai or lateral stage, and every command after it runs in post-processing on the returned rows. Only these commands work there: where, head, tail, sort, table, stats, dedup, rename, fields, top, rare, fillnull, eval, timechart, rex, risk, reverse, tree.

  • prevalence inside a subsearch (append, join, IN [ ... ]) is refused.

What's next

  • retro: sweep an indicator across history with a rarity verdict
  • risk: score rows, for example rare artifacts
  • stats: aggregate the filtered rows

On this page