On-Premise Log Collection
Run Vector agents and an aggregator to collect Windows events, Linux logs, syslog and files on your network and ship them to nano
Collect logs from your Windows endpoints, Linux servers, network devices, and applications with Vector. Forward them through an aggregator to nano so you can search local infrastructure from one place.
Use two tiers: agents on endpoints forward to a central aggregator that buffers and sends logs to nano. This keeps endpoint agents small and gives your firewall one egress point. A disk buffer holds events while nano is unreachable.
The agent and aggregator setup is the same on every deployment. The aggregator forwards each log type through its own HEC sink, authenticated with the ingest token bound to that type's nano source; the token decides the source.
| Self-run | Open core | |
|---|---|---|
| Aggregator sends to | https://<ingest-host> (HEC) | http://<nano-host>:8088 (HEC) |
| Auth | One ingest token per source | One ingest token per source |
| How the parser is chosen | The token's source | The token's source |
See Deployment models for the full endpoint list.
Choose a collection topology
| Concern | Agent only (direct to nano) | Agent + aggregator |
|---|---|---|
| Firewall rules | Every endpoint needs outbound access to nano | Only the aggregator does |
| Buffering | Each endpoint buffers on its own | One central disk buffer |
| Credentials | Tokens on every endpoint | Tokens only on the aggregator |
| Monitoring | Every endpoint | One aggregator |
For a handful of endpoints, agents can ship straight to nano: give each agent the aggregator's nano sink from Part 3 instead of the vector sink.
What you can collect
| Source | Vector source type | Typical use |
|---|---|---|
| Windows Event Log | windows_event_log | Security, Sysmon, PowerShell, Defender, Application, System |
| Linux logs | journald or file | Auth logs, auditd, systemd services |
| Log files | file | Application and web server logs |
| Network syslog | syslog | Firewalls, switches, routers, appliances |
| OTLP | opentelemetry | Application logs |
| Fluent protocol | fluent | Fluent Bit and Fluentd forwarders |
This guide starts with the most common case, Windows Event Log, then extends the pattern.
Part 1: Windows Event Log agent
The Vector agent runs as a Windows service on each endpoint, reads the Windows Event Log API and forwards to the aggregator.
Channels collected
| Channel | What it captures |
|---|---|
| Security | Logon and logoff, privilege use, object access, policy changes |
| System | Service start and stop, driver loads, time changes |
| Application | Crashes, warnings, installer events |
| Sysmon | Process creation, network connections, file and registry activity, DNS queries |
| PowerShell | Script block and module logging |
| Windows Defender | Detections, quarantine actions, scans |
Install Vector on Windows
# Download Vector (a release with the windows_event_log source)
Invoke-WebRequest -Uri "https://packages.timber.io/vector/latest/vector-x64.msi" -OutFile "$env:TEMP\vector.msi"
# Install silently
Start-Process msiexec.exe -Wait -ArgumentList "/i $env:TEMP\vector.msi /quiet /norestart"
# Verify
& "C:\Program Files\Vector\bin\vector.exe" --versionVector installs to C:\Program Files\Vector and creates a Windows service called vector.
Agent configuration
Create C:\ProgramData\Vector\vector.toml:
# Vector agent: Windows Event Log → aggregator at <AGGREGATOR_IP>:9000
data_dir = "C:/ProgramData/Vector/data"
[sources.windows_events]
type = "windows_event_log"
channels = [
"Application",
"System",
"Security",
"Microsoft-Windows-Sysmon/Operational",
"Microsoft-Windows-PowerShell/Operational",
"Microsoft-Windows-Windows Defender/Operational"
]
# Don't backfill existing events on first run (true for an initial import)
read_existing_events = false
# Per-channel rate limit, so an event storm can't swamp the agent
events_per_second = 500
# Truncate very long values (command lines, script blocks)
max_event_data_length = 2000
# Tag each event with the nano source it belongs to.
# Sysmon goes to the Sysmon parser, everything else to the Windows event parser.
[transforms.tag_source]
type = "remap"
inputs = ["windows_events"]
source = '''
channel = downcase(to_string(.channel) ?? "")
.source_type = if contains(channel, "sysmon") { "windows_sysmon" } else { "windows_event" }
'''
[sinks.aggregator]
type = "vector"
inputs = ["tag_source"]
address = "AGGREGATOR_IP:9000"
acknowledgements.enabled = true
# Disk buffer: survives agent restarts
[sinks.aggregator.buffer]
type = "disk"
max_size = 536870912 # 512 MB
when_full = "block"Replace AGGREGATOR_IP with the aggregator's address.
windows_event and windows_sysmon are the source types the bundled OCSF parsers claim (see Supported sources). The .source_type field is how the aggregator tells nano which parser to use; see Push vs pull routing.
Start the agent
# Validate the config
& "C:\Program Files\Vector\bin\vector.exe" validate "C:\ProgramData\Vector\vector.toml"
# Start the service and check it
Start-Service vector
Get-Service vectorTuning
| Parameter | Default here | When to change |
|---|---|---|
events_per_second | 500 | Raise on busy domain controllers; lower on workstations to save CPU. |
max_event_data_length | 2000 | Raise for full PowerShell script blocks or very long command lines. |
read_existing_events | false | true for a one-time historical import, then back to false. |
buffer.max_size | 512 MB | Raise if the aggregator may be offline for long periods. |
Part 2: Aggregator
The aggregator is a Linux host (VM, bare metal or container) that receives from all agents and forwards to nano. Place it where agents can reach it on :9000. It needs one outbound connection, to nano.
Prerequisites
- A Linux host with Docker and Docker Compose
- Inbound
:9000from agents - Outbound to nano: HTTPS
443to the ingest host (Self-run), or your nano host's HEC port8088(open core) - At least 2 GB of disk for buffering
Aggregator configuration
sudo mkdir -p /opt/vector-aggregator/config
cd /opt/vector-aggregatorCreate /opt/vector-aggregator/config/vector.toml. This part receives from agents and exposes metrics; the sink to nano comes from Part 3.
data_dir = "/var/lib/vector"
[api]
enabled = true
address = "0.0.0.0:8686"
# Receive from agents
[sources.agents]
type = "vector"
address = "0.0.0.0:9000"
acknowledgements.enabled = true
# Prometheus metrics for monitoring
[sources.internal_metrics]
type = "internal_metrics"
scrape_interval_secs = 10
[sinks.prometheus_metrics]
type = "prometheus_exporter"
inputs = ["internal_metrics"]
address = "0.0.0.0:9598"
default_namespace = "aggregator"
# Add the nano sink from Part 3 here, with inputs = ["agents"]Docker Compose
Create /opt/vector-aggregator/docker-compose.yml:
services:
vector-aggregator:
image: timberio/vector:latest-alpine
container_name: vector-aggregator
security_opt:
- no-new-privileges:true
ports:
- "9000:9000" # agents
- "9598:9598" # Prometheus metrics
- "8686:8686" # Vector API (restrict to admin IPs)
volumes:
- ./config/vector.toml:/etc/vector/vector.toml:ro
- vector-aggregator-data:/var/lib/vector
command: ["--config", "/etc/vector/vector.toml"]
restart: unless-stopped
volumes:
vector-aggregator-data:Start the aggregator
cd /opt/vector-aggregator
docker run --rm -v $(pwd)/config:/etc/vector:ro timberio/vector:latest-alpine validate /etc/vector/vector.toml
docker compose up -d
docker compose logs -fPart 3: Send from the aggregator to nano
Issue one ingest token per source under Ingestion → Log Sources → Credentials, here for windows_event and windows_sysmon, and export each as an environment variable for Vector. Split the agents' events by the .source_type they set, and send each log type through its own HEC sink with a disk buffer:
[transforms.by_source]
type = "route"
inputs = ["agents"]
route.windows_event = '.source_type == "windows_event"'
route.windows_sysmon = '.source_type == "windows_sysmon"'
[sinks.nano_windows_event]
type = "splunk_hec_logs"
inputs = ["by_source.windows_event"]
endpoint = "https://<ingest-host>"
default_token = "${NANO_TOKEN_WINDOWS_EVENT}"
endpoint_target = "event"
compression = "gzip"
encoding.codec = "json"
acknowledgements.enabled = true
[sinks.nano_windows_event.buffer]
type = "disk"
max_size = 1073741824 # 1 GB
when_full = "block"
[sinks.nano_windows_sysmon]
type = "splunk_hec_logs"
inputs = ["by_source.windows_sysmon"]
endpoint = "https://<ingest-host>"
default_token = "${NANO_TOKEN_WINDOWS_SYSMON}"
endpoint_target = "event"
compression = "gzip"
encoding.codec = "json"
acknowledgements.enabled = true
[sinks.nano_windows_sysmon.buffer]
type = "disk"
max_size = 1073741824 # 1 GB
when_full = "block"endpoint is the base URL; Vector's HEC sink adds /services/collector/event itself. On open core, use http://NANO_HOST:8088.
nano also accepts the Vector native protocol on :6000. From v2.0.30 it requires mutual TLS: each sender needs a client certificate from scripts/nano vector-client-cert, and its events land in the vector_native source unless nano trusts the declared source type. See Vector native. (On v2.0.29 and earlier it was plaintext with no credential check, on 127.0.0.1 by default.)
Firewall rules
| Direction | Port | Purpose |
|---|---|---|
| Inbound | 9000/tcp | Agents send events |
| Inbound | 9598/tcp | Prometheus scraping |
| Inbound | 8686/tcp | Vector API (optional; admin IPs only) |
| Outbound | 443/tcp to the ingest host (Self-run) or 8088/tcp (open core) | Forward to nano |
| Outbound | 6000/tcp to nano's ingest servers | Only with a Vector native sink (mutual TLS) |
Part 4: What happens in nano
- Routing. The sink's token decides each event's source, and that source's parser runs.
- Parsing. The parser maps the event to an OCSF object: a Windows logon becomes an Authentication event (
class_uid3002), a Sysmon process creation becomes Process Activity (1007). Source fields with no OCSF home are kept underunmapped, and the original line is kept inraw_data. - Storage. The event is written to ClickHouse with
metadata.log_sourceset to its source.
Deploy the parsers in Ingestion → Log Sources. The bundled OCSF parsers cover Windows Event Log, Sysmon, Linux auditd and journald, syslog and the common firewalls. For anything else, add a log source from sample events and build a parser for it.
Part 5: Operations
Health checks
# Aggregator: are events flowing?
curl -s http://aggregator-ip:9598/metrics | grep component_received_events_total
# Aggregator: is the buffer draining? (should be near 0)
curl -s http://aggregator-ip:9598/metrics | grep buffer_events
# Aggregator: Vector API health
curl -s http://aggregator-ip:8686/healthThen confirm the data in nano:
metadata.log_source="windows_event" | stats count by device.hostnamemetadata.log_source="windows_sysmon" class_uid=1007 | stats count by device.hostname, process.nameConfiguration management
For more than a few endpoints, deploy agents with your configuration management tool (Ansible, Intune, GPO startup scripts). Template the agent config with the aggregator address and restart the vector service when it changes.
Adding channels
Add channels to the agent's channels list (for example "DNS Server" or "Microsoft-Windows-DHCP-Server/Operational") and restart the service. Make sure a parser claims the source_type you tag them with.
Upgrading Vector
# Windows agents
Stop-Service vector
Start-Process msiexec.exe -Wait -ArgumentList "/i $env:TEMP\vector-new.msi /quiet /norestart"
Start-Service vector# Aggregator
cd /opt/vector-aggregator && docker compose pull && docker compose up -dPart 6: Other on-premise sources
Linux servers
Run a Vector agent with the journald or file source and tag events with a source a parser claims, for example linux_journald or linux_auditd:
data_dir = "/var/lib/vector"
[sources.journald]
type = "journald"
[sources.auditd]
type = "file"
include = ["/var/log/audit/audit.log"]
[transforms.tag_journald]
type = "remap"
inputs = ["journald"]
source = '.source_type = "linux_journald"'
[transforms.tag_auditd]
type = "remap"
inputs = ["auditd"]
source = '.source_type = "linux_auditd"'
[sinks.aggregator]
type = "vector"
inputs = ["tag_journald", "tag_auditd"]
address = "AGGREGATOR_IP:9000"
acknowledgements.enabled = trueSyslog from network devices
nano's ingest endpoints don't receive syslog (open core ships its syslog listener disabled), so receive syslog on the aggregator and tag it there. See Vector aggregator for syslog, OTLP, Fluent and file sources, and how to tag them.
Troubleshoot
Need help with these checks? Contact support with the deployment, request ID and time with timezone.
Agent not sending
- Check the service:
Get-Service vector(Windows) orsystemctl status vector(Linux). - Test the network:
Test-NetConnection -ComputerName AGGREGATOR_IP -Port 9000. - Validate the config with
vector validate.
Aggregator not forwarding
docker compose logs --tail 100for sink errors.- Check
component_received_events_totalat:9598/metricsto confirm agents are sending. - If
buffer_eventskeeps growing, the aggregator can't deliver to nano. Check the endpoint, port and token, and look for401(wrong, revoked or unbound token).
Events arrive but aren't parsed
- Check that each sink's token is bound to the right source, and that the source's parser is published.
- Look at the class breakdown. A parser that can't read an event emits it as Base Event (
class_uid0), so a source dominated by class 0 usually means a format mismatch:
metadata.log_source="windows_event" | stats count by class_uidWhat's next
- Vector aggregator: syslog, OTLP, Fluent and file sources
- Detection rules for your on-premise logs
- Deployment models: what each deployment accepts