Azure Event Hubs
Collect Azure Monitor and Entra logs through Event Hubs, with checkpoints in a blob container you control
Collect Azure Monitor logs through Event Hubs. Vector in your nano deployment reads the hub with an identity you control and checkpoints its read position in an Azure Blob Storage container you own, so a restart neither loses nor replays events.
Prerequisites
source_configs:create, pluscredentials:createandcredentials:useif you store a credential.- The
azure_monitorlog source enabled and its parser published under Log Sources. - An Event Hubs namespace and hub, and a storage account for checkpoints. A checkpoint store is mandatory.
- Network rules that let your nano deployment reach the hub on AMQPS port 5671 and the checkpoint storage on HTTPS port 443.
1. Choose how Vector authenticates
| Option | What to set up | Credential in nano |
|---|---|---|
| Entra app (client secret) | An app registration and service principal with Azure Event Hubs Data Receiver on the hub and Storage Blob Data Contributor on the checkpoint container | An azure_event_hubs credential with the directory (tenant) ID, client ID and client secret |
| Connection strings | A SAS policy with Listen on the hub, and the storage account's connection string | An azure_event_hubs credential with both connection strings |
| Managed identity | A managed identity on the nodes that run Vector, with the same two data roles | None: leave the credential empty |
Scope the roles to the hub and the container, not the namespace or subscription:
az role assignment create --assignee <APP_OR_IDENTITY_ID> --role "Azure Event Hubs Data Receiver" \
--scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.EventHub/namespaces/<NAMESPACE>/eventhubs/<EVENT_HUB>
az role assignment create --assignee <APP_OR_IDENTITY_ID> --role "Storage Blob Data Contributor" \
--scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<STORAGE_RESOURCE_GROUP>/providers/Microsoft.Storage/storageAccounts/<CHECKPOINT_ACCOUNT>/blobServices/default/containers/<CHECKPOINT_CONTAINER>Secrets live only in the sealed credential, never in the pull source's connection settings.
2. Create checkpoint storage
In the storage account, create a blob container for checkpoints. Vector writes standard Event Hubs ownership and checkpoint blobs there every 5 seconds, and only for events already delivered.
az storage container create --account-name <CHECKPOINT_ACCOUNT> --name <CHECKPOINT_CONTAINER> --auth-mode login3. Create a dedicated consumer group
In the event hub's Consumer groups, create nano. Other readers using the same group compete for partitions, so dedicate this group to nano.
az eventhubs eventhub consumer-group create --resource-group <RESOURCE_GROUP> \
--namespace-name <NAMESPACE> --eventhub-name <EVENT_HUB> --name nanoLeaving the pull source's consumer group empty uses $Default; enter nano to use the dedicated group.
4. Stream logs to the hub
For subscription Activity Logs, open Monitor → Activity log → Export Activity Logs. For Entra logs, open Entra ID → Diagnostic settings and select categories such as SignInLogs and AuditLogs. For resource logs, open the resource's diagnostic settings. Select Stream to an event hub and choose the namespace and hub above.
For a resource:
az monitor diagnostic-settings create --name nano-pull --resource <RESOURCE_ID> --event-hub <EVENT_HUB> \
--event-hub-rule /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.EventHub/namespaces/<NAMESPACE>/authorizationRules/RootManageSharedAccessKey \
--logs '[{"categoryGroup":"allLogs","enabled":true}]'Use this command for resources that support the allLogs category group; use the console's available categories for Activity Logs or Entra exports. The authorization rule lets Azure Monitor write to the hub. nano doesn't need its key. Azure Monitor sends a {"records": [...]} envelope; nano unpacks it into one event per record (at most 10,000 per message; a larger batch adds one truncation event).
5. Add the pull source in nano
Open Ingestion → Log Sources → Cloud pull. If you use a credential, add it under Credentials first. Then choose Add cloud pull, pick Azure Event Hubs, and enter:
| Field | What to enter |
|---|---|
| Event Hub | The hub name. |
| Consumer group | nano, or your dedicated group. Empty means $Default. |
| Namespace | The namespace name (acme-logs) or host (acme-logs.servicebus.windows.net), not a connection string. |
| Checkpoint container | The container name. |
| Storage account | The checkpoint storage account name, not its URL. |
| Credential | Your azure_event_hubs credential, or none for managed identity. |
Route it to the azure_monitor log source and Deploy it.
Verify
Generate real activity covered by the diagnostic setting. In Search, select the corresponding time range and count events by their resulting source:
metadata.log_source=azure_* | stats count by metadata.log_sourceThe azure_monitor parser dispatches mixed categories per record, so the resulting source can be a specific Azure parser rather than azure_monitor. Inspect those rows and use their metadata.log_source value in a narrower query. See nPL search syntax and stats.
Troubleshoot
Need help with these checks? Contact support with the deployment, request ID and time with timezone.
Authentication or authorization errors in Vector logs
Check the directory and client IDs and the secret, or the connection strings. Confirm Azure Event Hubs Data Receiver on the hub and Storage Blob Data Contributor on the checkpoint container for the app or managed identity.
The checkpoint container is not found
Create the container in the named storage account, or correct the container name on the pull source.
Deployed but empty
Check diagnostic categories and destination, incoming hub traffic, the dedicated consumer group, the azure_monitor parser, and the search time range. Another reader on the same consumer group competes for partitions. A deployed source is not proof of data arrival.
Security and revocation
Grant roles on one hub and one checkpoint container only. Revoke access by removing the data-role assignments, rotating the client secret, or regenerating the SAS keys, and undeploy or delete the pull source in nano.