nano SIEM
SOAR

AI Judge

Supported in:Self-hosted

Ask an AI model a fixed question about evidence a flow already holds, branch on its labeled verdict, and control what it costs with token caps, caching, and a daily budget.

Use a Judge step to let an AI model classify evidence before the flow decides what to do. You write a fixed question and a set of labels, and choose which values from the trigger and earlier steps the model sees. The model answers with one of your labels, a confidence, and a short reason.

Add a Judge step

Add the block

Drag Judge from the block palette onto the flow.

Write the question

Write one fixed question, for example "Is this sign-in consistent with the user's normal behavior?". The question can't contain values from the run.

Set the labels

Add the labels the model must choose from, for example benign, suspicious, and malicious.

Choose the evidence

Pick the values the model should see, such as alert.entities.user or the output of an earlier search step. Optionally set an entity, a single value the verdict is about.

Branch on the verdict

Add a Condition after the Judge step that reads results.<step id>.verdict, for example results.judge.verdict = "malicious".

What the model gets

The model gets only:

  • your question;
  • your labels;
  • the evidence values and the entity you chose, with the path each one came from.

nano passes the evidence as data, inside a fenced block, and tells the model to ignore any instructions in it. The model gets no tools, can't run searches, and can't see anything you didn't select.

What the step returns

OutputContents
results.<id>.verdictOne of your labels, or unknown
results.<id>.confidence0 to 1. Always 0 for unknown.
results.<id>.whyThe model's reason, up to 1,024 characters
results.<id>.tokensTokens counted for this call. 0 for a cached verdict.
results.<id>.cachedtrue when the verdict came from the cache

If the model's reply isn't a JSON verdict, or names a label you didn't define, the verdict is unknown with confidence 0. nano never guesses a label.

Limits

SettingLimit
QuestionUp to 2,048 characters
Labels1 to 32. Letters, digits, _, and -, up to 64 characters each, unique ignoring case. unknown is reserved.
Evidence1 to 32 values, at most 32 KiB together
EntityOne value (text, number, or true/false), at most 1,024 bytes
Token cap2,000 by default, up to 8,000 for each call
Timeout60 seconds
Cache6 hours by default, from 1 second up to 24 hours

Cost controls

The token cap

Before each call, nano reserves the step's token cap from the daily budget. After the call, it returns the part it didn't use. Your AI provider doesn't report token counts to nano here, so nano estimates the tokens used from the length of the question, evidence, and reply, and counts at most the cap.

The cache

nano caches each verdict for the step's cache time. The cache key covers the question, labels, entity, evidence, and token cap, so the same question about the same evidence returns the cached verdict and uses no tokens.

The daily token budget

All Judge steps on your deployment share one daily budget. Set it at Settings → SOAR → AI Judge daily token budget, from 0 to 1,000,000,000 tokens. The default is 250,000. Changing it needs settings:system.

When a call's token cap doesn't fit in what is left of the day's budget, nano doesn't call the model. The step fails with judge_budget_exhausted and the run fails. A budget of 0 stops all Judge calls. Cached verdicts still work.

How Judge results are recorded

Each Judge call is recorded as a step of its run: the verdict, confidence, reason, tokens, and whether it was cached. Open the run on Automation → Runs to see it. Judge calls in tests use the budget too.

Troubleshoot

The step fails with judge_budget_exhausted

The daily budget is used up. Raise AI Judge daily token budget, lower the step's token cap, or wait for the next day. Check which flows run Judge steps most often on Automation → Runs.

The step fails with judge_timeout or judge_provider_error

The AI provider didn't answer within 60 seconds, or returned an error. Check the provider in AI & pivt. nano returns the reserved tokens to the budget.

The verdict is always unknown

The evidence doesn't decide the question, or the model isn't answering in the expected format. Make the question narrower, check that the evidence paths hold values for this trigger, and give labels that cover the possible answers.

What's next

On this page