prevalence
Filter or annotate results by how many of your hosts have seen a hash, domain or IP: find the rare and the brand new.
Use prevalence to filter or annotate your results by how common a file hash, domain, or public IP is across your environment. You can use rare or newly seen artifacts as investigation leads.
Prevalence is per deployment: counts come only from your own events.
Syntax
... | prevalence <condition> [[AND] <condition> ...] [window=24h|7d|30d] [enrich=true]
... | prevalence enrich=true [window=24h|7d|30d]
... | prevalence <field>[, <field> ...] [window=24h|7d|30d]Filter form
Keep only rows whose artifacts satisfy every condition.
| Condition field | Compares | Value |
|---|---|---|
hash_prevalence | Number of hosts that have seen the row's file or process hash | an integer |
domain_prevalence | Number of hosts that have contacted the row's domain | an integer |
hash_first_seen | When the hash was first seen in your environment | now() - <n>[s|m|h|d], or now() - INTERVAL <n> <UNIT> |
domain_first_seen | When the domain was first seen | same as above |
Operators: <, <=, >, >=, =, !=. A bare number after now() - is seconds.
hash_first_seen > now() - 24h means "first seen within the last 24 hours".
Enrich form
prevalence enrich=true keeps every row and adds prevalence columns, including:
| Column | Meaning |
|---|---|
host_count | Hosts that have seen the artifact (the lowest count when the row has several artifacts) |
is_rare | true when the artifact is at or below your rarity threshold |
total_occurrences | Total events that carried the artifact |
first_seen, last_seen | First and last time the artifact was seen |
prevalence_score | 0–100 prevalence score |
prevalence_artifact | The artifact value(s) checked, comma-separated |
prevalence_type | The artifact type(s): domain, ip, hash |
prevalence_min | Same as host_count |
prevalence_first_seen, prevalence_last_seen | Same as first_seen, last_seen |
The positional form prevalence <field>[, <field>] enriches the values of the named fields only, for example prevalence http_request.url.hostname. It adds the same columns and does not filter.
Options
| Option | Default | Description |
|---|---|---|
window | 30d | Look-back for the prevalence counts: 24h, 7d or 30d. interval= is an alias. window=1h is rejected: "prevalence window=1h is not supported; use 24h, 7d, or 30d". |
enrich | false (filter form) | With the filter form, also add the enrich columns to the rows that pass. |
threshold= | — | Accepted on the positional form and ignored. |
Private IP addresses are not counted as artifacts.
Examples
Process executions whose hash three or fewer hosts have run in the last week:
class_uid=1007 | prevalence hash_prevalence <= 3 window=7d | table time, device.hostname, actor.user.name, process.name, process.file.hashes.valueHashes first seen in your environment within the last 24 hours:
class_uid=1007 | prevalence hash_first_seen > now() - 24h | stats count by process.name, process.file.hashes.valueWeb traffic to rare domains that are also new this week:
class_uid=4002 | prevalence domain_prevalence < 5 AND domain_first_seen > now() - 7d | stats count by http_request.url.hostname, src_endpoint.ipAnnotate proxy traffic and keep the rare destinations:
metadata.log_source=conduit_proxy | prevalence enrich=true | where is_rare=true | stats count by http_request.url.hostname | sort -countAnnotate one field only:
class_uid=4002 | prevalence http_request.url.hostname | table time, src_endpoint.ip, http_request.url.hostname, host_count, first_seenPrevalence stamped at ingest
Events also carry the prevalence of their artifacts as of ingest in the OCSF enrichments array: entries with name prevalence, provider nano and type reputation. short_desc is new artifact, rare artifact or observed artifact, and data holds host_count, is_rare, is_new, first_seen and the other counts. These are normal fields, so you can filter on them in the base search without the prevalence command:
metadata.log_source=conduit_proxy enrichments.short_desc="new artifact" | stats count by http_request.url.hostnameThe ingest stamp answers "was this rare when it happened?" and never changes. | prevalence answers "how rare is it now?". See OCSF schema for the enrichments array.
Run commands after enrichment
prevalence is never compiled into the ClickHouse query. The search splits at the first lookup, prevalence, inputlookup, resolve_identity, ai or lateral stage, and every command after it runs in post-processing on the returned rows. Only these commands work there: where, head, tail, sort, table, stats, dedup, rename, fields, top, rare, fillnull, eval, timechart, rex, risk, reverse, tree.
prevalenceinside a subsearch (append,join,IN [ ... ]) is refused.