nano SIEM
User Guide

Data Management

Supported in:Self-hosted

Create lookup tables from files or by hand, refresh them with scheduled URL ingestion, and use them with lookup, inputlookup and ioc in lookup().

Create lookup tables to join reference data to your events, such as asset inventories, user directories, allowlists, and threat indicators. Load a file, define a schema, edit rows, or refresh the table on a URL schedule.

Lookup tables are for reference data, not events. Events reach nano through your log sources; there's no file upload into the event store.

Lookup tables

Rules → Lookup Tables (/rules/lookup-tables) lists your tables. You need lookup:view to see them, lookup:create to create one, lookup:edit to change rows or ingestion, and lookup:delete to delete one.

Create a table

Open Rules → Lookup Tables and select New. Choose one of three modes:

  • Upload a file: load a CSV, JSON or NDJSON file once.
  • Define a schema: declare the columns, then add rows by hand or through the API.
  • Import from URL: fetch a file now and, optionally, on a schedule (see Scheduled URL ingestion).

Create from a file

  1. Open Rules → Lookup Tables, select New, then Upload a file.
  2. Choose a CSV, JSON or NDJSON file.
  3. Set the table name, an optional description and an optional primary key column.
  4. Review the detected column types and override any that are wrong.
  5. Create the table.
FormatsCSV and TSV (.csv, .tsv), JSON array of objects (.json), NDJSON (.ndjson, .jsonl)
CSV optionsDelimiter (default ,), header row (on by default, or detected), custom headers
JSON optionFlatten nested objects
Column typestext, integer, float, boolean, timestamp, json
File sizeUp to 100 MB
Rows per tableUp to 1,000,000

Column names are sanitized to safe identifiers. The create response lists any columns that were renamed.

Create from a schema

Define the columns and add rows later. Through the API: POST /api/lookup-tables/schema with name, optional description, columns (each name, data_type, nullable) and optional primary_key.

Edit rows

Open a table to page through its rows, add rows, edit a row or delete rows. Through the API, one request adds up to 1,000 rows.

Replace or append

When you load data into an existing table, choose a mode:

  • Replace (default): the new data replaces all existing rows. Use it for a full refresh from an authoritative source.
  • Append: the new rows are added to the existing ones.

Usage and history

Each table shows where it's used (/usage) and an ingestion history of loads and edits (/ingestion-history).

Scheduled URL ingestion

A table can refresh itself from a URL on a cron schedule. Set this up with Import from URL when you create the table, or later from the table's page:

FieldMeaning
URLWhere to fetch the file.
ScheduleA five-field cron expression, or a preset (every hour, every 6 hours, daily at midnight or 6 AM, weekly on Sunday, monthly on the 1st).
Formatcsv, json or ndjson, with the same parsing options as an upload.
Modereplace or append.
Auth headersOptional request headers, such as Authorization. Stored values are encrypted and shown masked.
Retry policymax_retries (default 3) and retry_delay_secs (default 60).
EnabledTurn the schedule on or off without deleting it.

You can trigger a fetch outside the schedule. The table shows the result of the last run, and the ingestion history records every run.

A scheduled fetch can reach any public internet host over HTTPS or HTTP. Private, loopback, link-local and cloud-metadata addresses are refused when you save the schedule. A fetch that can't connect is recorded as failed in the ingestion history with the connection error.

Use lookups in searches

lookup: add fields to events

lookup <table> <column> [AS <event_field>] [OUTPUT <col> [AS <name>], ...] [CASE_INSENSITIVE]

<column> is the table's key column. AS <event_field> names the event field to match it against, which is usually an OCSF path. OUTPUT picks the columns to add (all columns if you leave it out). Full syntax: lookup.

Add the owner of each destination host:

class_uid=4001 | lookup asset_inventory ip AS dst_endpoint.ip OUTPUT owner, department

Enrich logons with directory data, then filter on it:

class_uid=3002 | lookup user_directory username AS user.name OUTPUT department, manager | where department="Finance"

Match domains case-insensitively and keep only hits:

class_uid=4003 | lookup threat_domains domain AS query.hostname OUTPUT threat_type CASE_INSENSITIVE | where isnotnull(threat_type)

Filter before the lookup, so fewer rows are looked up:

class_uid=4001 | where NOT is_private_ip(dst_endpoint.ip) | lookup ip_reputation ip AS dst_endpoint.ip OUTPUT reputation | stats count by reputation

inputlookup: read a table as results

| inputlookup threat_domains

Use it to review a table, or as the source of a pipeline. See inputlookup.

inputlookup can also fetch a remote file with url="https://…" from any public URL your deployment can reach; private, loopback and cloud-metadata addresses are blocked. Options: format, key, timeout (1–60 s, default 30), max_rows (default 10,000, max 100,000) and cache_ttl (0–3,600 s, default 300).

ioc in lookup(): sweep a table's values across your events

ioc in lookup("threat_domains") | stats count by metadata.log_source

This uses every value in the table as an indicator and finds events that contain any of them. Add a column name to use one column: lookup("threat_domains", "domain"). Use it in the base search, not in a subsearch or after a pipe. See retro.

For curated indicator sets, use Rules → IOC Lists instead of a plain lookup table. See Enrichments.

Lookup table API

Method and pathPermissionPurpose
GET /api/lookup-tableslookup:viewList tables
POST /api/lookup-tableslookup:createCreate from a file (multipart: file and a JSON config)
POST /api/lookup-tables/schemalookup:createCreate an empty table from a schema
GET /api/lookup-tables/{name}lookup:viewTable details
DELETE /api/lookup-tables/{name}lookup:deleteDelete a table
GET /api/lookup-tables/{name}/samplelookup:viewSample rows
GET /api/lookup-tables/{name}/rowslookup:viewPage through rows (page, page_size)
POST /api/lookup-tables/{name}/rowslookup:editAdd rows (up to 1,000 per request)
DELETE /api/lookup-tables/{name}/rowslookup:editDelete rows by id
PUT, DELETE /api/lookup-tables/{name}/rows/{row_id}lookup:editEdit or delete one row
POST /api/lookup-tables/querylookup:viewLook up one key value, or many
GET /api/lookup-tables/{name}/usagelookup:viewWhere the table is used
GET /api/lookup-tables/{name}/ingestion-historylookup:viewLoad and edit history
GET, PUT, DELETE /api/lookup-tables/{name}/ingestionlookup:view / lookup:editRead, set or remove scheduled ingestion
POST /api/lookup-tables/{name}/ingestion/triggerlookup:editRun ingestion now
POST /api/lookup-tables/{name}/ingestion/enable, …/disablelookup:editTurn the schedule on or off
POST /api/lookup-tables/validate-cronlookup:viewCheck a cron expression and preview next runs

Create a table from a CSV:

curl -X POST "https://your-instance/api/lookup-tables" \
  -H "X-API-Key: $NANO_API_KEY" \
  -F "file=@asset_inventory.csv" \
  -F 'config={"name": "asset_inventory", "description": "CMDB export", "primary_key": "ip", "format": "csv", "mode": "replace"}'

Look up several keys at once:

curl -X POST "https://your-instance/api/lookup-tables/query" \
  -H "X-API-Key: $NANO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"table_name": "threat_domains", "key_field": "domain", "key_values": ["bad.example", "evil.example"], "output_fields": ["domain", "threat_type"], "case_insensitive": true}'

Schedule a daily refresh:

curl -X PUT "https://your-instance/api/lookup-tables/threat_domains/ingestion" \
  -H "X-API-Key: $NANO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://feeds.example.com/domains.csv",
    "cron_expression": "0 6 * * *",
    "parser_config": {"format": "csv", "csv_has_headers": true},
    "mode": "replace",
    "retry_policy": {"max_retries": 3, "retry_delay_secs": 60},
    "enabled": true
  }'

Practices

  • Name tables for what they hold (asset_inventory, vip_users) and set a primary key on the column you join on.
  • Prefer replace for authoritative sources so deleted entries disappear; use append for accumulating lists.
  • Keep lookups late and narrow. Filter first, then look up, and list only the OUTPUT columns you need.
  • Check ingestion history after changing a feed URL or format.

Troubleshoot

Need help with these checks? Contact support with the affected deployment, request ID and time with timezone.

A lookup pipeline rejects a command

Check the supported commands listed under Commands after lookup. Move other stages before lookup; a lookup cannot run inside a subsearch.

Scheduled URL ingestion fails

Check ingestion history for the connection error. Scheduled ingestion refuses private, loopback, link-local, and cloud-metadata addresses.

What's next

On this page