Data Management
Create lookup tables from files or by hand, refresh them with scheduled URL ingestion, and use them with lookup, inputlookup and ioc in lookup().
Create lookup tables to join reference data to your events, such as asset inventories, user directories, allowlists, and threat indicators. Load a file, define a schema, edit rows, or refresh the table on a URL schedule.
Lookup tables are for reference data, not events. Events reach nano through your log sources; there's no file upload into the event store.
Lookup tables
Rules → Lookup Tables (/rules/lookup-tables) lists your tables. You need lookup:view to see them, lookup:create to create one, lookup:edit to change rows or ingestion, and lookup:delete to delete one.
Create a table
Open Rules → Lookup Tables and select New. Choose one of three modes:
- Upload a file: load a CSV, JSON or NDJSON file once.
- Define a schema: declare the columns, then add rows by hand or through the API.
- Import from URL: fetch a file now and, optionally, on a schedule (see Scheduled URL ingestion).
Create from a file
- Open Rules → Lookup Tables, select New, then Upload a file.
- Choose a CSV, JSON or NDJSON file.
- Set the table name, an optional description and an optional primary key column.
- Review the detected column types and override any that are wrong.
- Create the table.
| Formats | CSV and TSV (.csv, .tsv), JSON array of objects (.json), NDJSON (.ndjson, .jsonl) |
| CSV options | Delimiter (default ,), header row (on by default, or detected), custom headers |
| JSON option | Flatten nested objects |
| Column types | text, integer, float, boolean, timestamp, json |
| File size | Up to 100 MB |
| Rows per table | Up to 1,000,000 |
Column names are sanitized to safe identifiers. The create response lists any columns that were renamed.
Create from a schema
Define the columns and add rows later. Through the API: POST /api/lookup-tables/schema with name, optional description, columns (each name, data_type, nullable) and optional primary_key.
Edit rows
Open a table to page through its rows, add rows, edit a row or delete rows. Through the API, one request adds up to 1,000 rows.
Replace or append
When you load data into an existing table, choose a mode:
- Replace (default): the new data replaces all existing rows. Use it for a full refresh from an authoritative source.
- Append: the new rows are added to the existing ones.
Usage and history
Each table shows where it's used (/usage) and an ingestion history of loads and edits (/ingestion-history).
Scheduled URL ingestion
A table can refresh itself from a URL on a cron schedule. Set this up with Import from URL when you create the table, or later from the table's page:
| Field | Meaning |
|---|---|
| URL | Where to fetch the file. |
| Schedule | A five-field cron expression, or a preset (every hour, every 6 hours, daily at midnight or 6 AM, weekly on Sunday, monthly on the 1st). |
| Format | csv, json or ndjson, with the same parsing options as an upload. |
| Mode | replace or append. |
| Auth headers | Optional request headers, such as Authorization. Stored values are encrypted and shown masked. |
| Retry policy | max_retries (default 3) and retry_delay_secs (default 60). |
| Enabled | Turn the schedule on or off without deleting it. |
You can trigger a fetch outside the schedule. The table shows the result of the last run, and the ingestion history records every run.
A scheduled fetch can reach any public internet host over HTTPS or HTTP. Private, loopback, link-local and cloud-metadata addresses are refused when you save the schedule. A fetch that can't connect is recorded as failed in the ingestion history with the connection error.
Use lookups in searches
lookup: add fields to events
lookup <table> <column> [AS <event_field>] [OUTPUT <col> [AS <name>], ...] [CASE_INSENSITIVE]<column> is the table's key column. AS <event_field> names the event field to match it against, which is usually an OCSF path. OUTPUT picks the columns to add (all columns if you leave it out). Full syntax: lookup.
Add the owner of each destination host:
class_uid=4001 | lookup asset_inventory ip AS dst_endpoint.ip OUTPUT owner, departmentEnrich logons with directory data, then filter on it:
class_uid=3002 | lookup user_directory username AS user.name OUTPUT department, manager | where department="Finance"Match domains case-insensitively and keep only hits:
class_uid=4003 | lookup threat_domains domain AS query.hostname OUTPUT threat_type CASE_INSENSITIVE | where isnotnull(threat_type)Filter before the lookup, so fewer rows are looked up:
class_uid=4001 | where NOT is_private_ip(dst_endpoint.ip) | lookup ip_reputation ip AS dst_endpoint.ip OUTPUT reputation | stats count by reputationinputlookup: read a table as results
| inputlookup threat_domainsUse it to review a table, or as the source of a pipeline. See inputlookup.
inputlookup can also fetch a remote file with url="https://…" from any public URL your deployment can reach; private, loopback and cloud-metadata addresses are blocked. Options: format, key, timeout (1–60 s, default 30), max_rows (default 10,000, max 100,000) and cache_ttl (0–3,600 s, default 300).
ioc in lookup(): sweep a table's values across your events
ioc in lookup("threat_domains") | stats count by metadata.log_sourceThis uses every value in the table as an indicator and finds events that contain any of them. Add a column name to use one column: lookup("threat_domains", "domain"). Use it in the base search, not in a subsearch or after a pipe. See retro.
For curated indicator sets, use Rules → IOC Lists instead of a plain lookup table. See Enrichments.
Lookup table API
| Method and path | Permission | Purpose |
|---|---|---|
GET /api/lookup-tables | lookup:view | List tables |
POST /api/lookup-tables | lookup:create | Create from a file (multipart: file and a JSON config) |
POST /api/lookup-tables/schema | lookup:create | Create an empty table from a schema |
GET /api/lookup-tables/{name} | lookup:view | Table details |
DELETE /api/lookup-tables/{name} | lookup:delete | Delete a table |
GET /api/lookup-tables/{name}/sample | lookup:view | Sample rows |
GET /api/lookup-tables/{name}/rows | lookup:view | Page through rows (page, page_size) |
POST /api/lookup-tables/{name}/rows | lookup:edit | Add rows (up to 1,000 per request) |
DELETE /api/lookup-tables/{name}/rows | lookup:edit | Delete rows by id |
PUT, DELETE /api/lookup-tables/{name}/rows/{row_id} | lookup:edit | Edit or delete one row |
POST /api/lookup-tables/query | lookup:view | Look up one key value, or many |
GET /api/lookup-tables/{name}/usage | lookup:view | Where the table is used |
GET /api/lookup-tables/{name}/ingestion-history | lookup:view | Load and edit history |
GET, PUT, DELETE /api/lookup-tables/{name}/ingestion | lookup:view / lookup:edit | Read, set or remove scheduled ingestion |
POST /api/lookup-tables/{name}/ingestion/trigger | lookup:edit | Run ingestion now |
POST /api/lookup-tables/{name}/ingestion/enable, …/disable | lookup:edit | Turn the schedule on or off |
POST /api/lookup-tables/validate-cron | lookup:view | Check a cron expression and preview next runs |
Create a table from a CSV:
curl -X POST "https://your-instance/api/lookup-tables" \
-H "X-API-Key: $NANO_API_KEY" \
-F "file=@asset_inventory.csv" \
-F 'config={"name": "asset_inventory", "description": "CMDB export", "primary_key": "ip", "format": "csv", "mode": "replace"}'Look up several keys at once:
curl -X POST "https://your-instance/api/lookup-tables/query" \
-H "X-API-Key: $NANO_API_KEY" \
-H "Content-Type: application/json" \
-d '{"table_name": "threat_domains", "key_field": "domain", "key_values": ["bad.example", "evil.example"], "output_fields": ["domain", "threat_type"], "case_insensitive": true}'Schedule a daily refresh:
curl -X PUT "https://your-instance/api/lookup-tables/threat_domains/ingestion" \
-H "X-API-Key: $NANO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://feeds.example.com/domains.csv",
"cron_expression": "0 6 * * *",
"parser_config": {"format": "csv", "csv_has_headers": true},
"mode": "replace",
"retry_policy": {"max_retries": 3, "retry_delay_secs": 60},
"enabled": true
}'Practices
- Name tables for what they hold (
asset_inventory,vip_users) and set a primary key on the column you join on. - Prefer replace for authoritative sources so deleted entries disappear; use append for accumulating lists.
- Keep lookups late and narrow. Filter first, then look up, and list only the
OUTPUTcolumns you need. - Check ingestion history after changing a feed URL or format.
Troubleshoot
Need help with these checks? Contact support with the affected deployment, request ID and time with timezone.
A lookup pipeline rejects a command
Check the supported commands listed under Commands after lookup. Move other stages before lookup; a lookup cannot run inside a subsearch.
Scheduled URL ingestion fails
Check ingestion history for the connection error. Scheduled ingestion refuses private, loopback, link-local, and cloud-metadata addresses.