nano SIEM
Search Commands

dedup

Supported in:Self-hosted

Keep one event per unique value, or combination of values, of the fields you name.

Use dedup to keep one row for each distinct value or combination of values in your chosen fields. uniq is an alias.

Syntax

... | dedup <field> [, <field> ...]
... | uniq <field> [, <field> ...]
ArgumentDescription
fieldOne or more fields that form the key. Separate them with commas or spaces.

For compatibility with other SIEM query languages, dedup also accepts a leading count (dedup 3 user.name), keepfirst= / keeplast= and a trailing sortby clause. nano parses them and ignores them.

How it works

  • For each key, dedup keeps the oldest event in the time range. Which event wins when two share the same time is not defined.
  • Events where a key field is empty are not dropped: they form their own group, and one of them is kept.
  • The output keeps every field of the surviving event. Results are ordered newest first, like any event search.
  • To keep the newest event per key instead, aggregate: stats latest(time) as last_seen by <field>. To count events per key, use stats.

Examples

The first logon seen for each user in the window:

class_uid=3002 status_id=1 | dedup user.name | table time, user.name, src_endpoint.ip, device.hostname

One row per user and source address pair:

class_uid=3002 status_id=2 | dedup user.name, src_endpoint.ip | table time, user.name, src_endpoint.ip

One example of each distinct command line on a host:

class_uid=1007 device.hostname="ws-hr-005.corp.local" | dedup process.cmd_line | table time, process.name, process.cmd_line

The first CloudTrail call of each API operation:

metadata.log_source=aws_cloudtrail | uniq api.operation | table time, api.operation, actor.user.name, src_endpoint.ip

Notes

  • dedup works on raw events and after other commands. Put filters before it so fewer events are grouped.
  • To get the distinct values only, stats count by <field> is cheaper and also tells you how many events each value has.

What's next

  • stats: count or summarize per key
  • top: most common values
  • sort: order the remaining rows

On this page