Suricata, Supercharged.

A powerful bolt-on toolkit that transforms Suricata into a capable NDR with AI-powered detection and automated response.

Architecture

Suricata EVE JSON ──► rockfish ingest ──► Parquet ──► S3 (optional)
                                              │
                          ┌───────────────────┼───────────────────┐
                          ▼                   ▼                   ▼
                       Report              Hunt                MCP/Chat
                    (HTML pages)      (threat detection)    (AI-native queries)
                                          │
                                          ▼
                                       Alert
                                  (MQTT / Kafka)

Core Pipeline

Ingest reads Suricata’s EVE JSON, partitions events by type (alert, flow, DNS, HTTP, TLS, SSH, SMTP, fileinfo, anomaly, DHCP, MQTT, Modbus, DNP3, and more), writes columnar Parquet with optional hive-style date partitioning, and flushes on configurable time or memory thresholds.

Hunt builds a communication graph from flow data and applies behavioral detection algorithms — beaconing, lateral movement, C2 fanout, port scanning, data exfiltration, DNS tunneling, and more. Findings are scored with HBOS or Isolation Forest anomaly models.

Report generates self-contained HTML dashboards with 12+ pages covering alerts, threats, DNS, TLS, flows, hosts, network topology, asset inventory, and hunt findings.

Alert reads detection events from Parquet, normalizes them into a common JSON payload, and publishes to MQTT and/or Kafka for automated response workflows.

MCP exposes Parquet data to AI assistants via the Model Context Protocol with query and hunt tools.

Chat provides a conversational AI interface for network security analysis with pluggable LLM backends.

Commands at a Glance

Technical Foundation

• Language: Rust — single static binary, no runtime dependencies
• Query Engine: Embedded DuckDB for analytical SQL on Parquet
• Storage Format: Apache Parquet with Zstd compression
• Protocols: MQTT, Kafka, MCP (Model Context Protocol)
• Crypto: Ed25519 license verification, TLS 1.3 transport
• Concurrency: Rayon parallel query execution, Tokio async I/O

Next Steps

• Installation - Build and deploy Rockfish NDR
• Quick Start - Get up and running in minutes
• Configuration - YAML configuration reference

Installation

Building from Source

Rockfish NDR is built with Rust. You need a working Rust toolchain (1.75+).

Prerequisites

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# DuckDB library (required)
# The DuckDB shared library must be available at build time.
# Set DUCKDB_LIB_DIR to the directory containing libduckdb.so
export DUCKDB_LIB_DIR=/usr/local/lib

Standard Build

# Build with default features (all commands enabled)
cargo build --release -p rockfish-cli

# The binary is at:
ls -la target/release/rockfish

Feature-Selective Build

# Build with only report and hunt
cargo build --release -p rockfish-cli --features report,hunt

# Build with Kafka support
cargo build --release -p rockfish-cli --features report,hunt,kafka

Available Features

Deployment

Copy Binary

# Deploy to standard location
cp target/release/rockfish /opt/rockfish/bin/rockfish

# Create configuration directories
mkdir -p /opt/rockfish/etc
mkdir -p /opt/rockfish/shared/extensions

Configuration File

# Copy or create configuration
cp rockfish.yaml /opt/rockfish/etc/rockfish.yaml

Rockfish searches for configuration in this order:

1. --config <path> (CLI argument)
2. ./rockfish.yaml
3. /etc/rockfish/rockfish.yaml
4. ~/.config/rockfish/rockfish.yaml

Environment File

Credentials and secrets are stored in an environment file:

# Create environment file
cat > /opt/rockfish/etc/rockfish.env << 'EOF'
ROCKFISH_S3_BUCKET=rockfish-data
ROCKFISH_S3_REGION=us-east-1
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
ABUSEIPDB_API_KEY=...
EOF

Docker Deployment

Production Image

Build and run Rockfish NDR in a container with all features:

# Build the image
docker build -t rockfish:latest -f Dockerfile .

# Run with configuration and data volumes
docker run -d \
  --name rockfish \
  -v /opt/rockfish/etc:/opt/rockfish/etc:ro \
  -v /data/rockfish:/data/rockfish \
  -p 3000:3000 \
  -p 8082:8082 \
  rockfish:latest ingest --socket /var/run/suricata/eve.sock

The production image includes all default features plus Kafka support, with DuckDB bundled from source.

Demo Report Image

Generate a self-contained demo report served by nginx:

# Build the demo image
docker build -t rockfish-demo:latest -f Dockerfile.demo .

# Run on port 8080
docker run -d --name rockfish-demo -p 8080:8080 rockfish-demo:latest

Open http://localhost:8080 to view the demo report.

Verify Installation

# Check version
rockfish --version

# Show configuration and features
rockfish config

Next Steps

• Quick Start - Ingest, hunt, and report in minutes
• Configuration - Full YAML configuration reference

Quick Start

This guide walks you through ingesting Suricata logs, running threat detection, generating a report, and publishing alerts.

1. Ingest Suricata Logs

From a File

# Ingest EVE JSON into Parquet
rockfish ingest -i /var/log/suricata/eve.json \
  -o /data/rockfish --sensor my-sensor --hive

Continuous Ingestion

# Follow mode — tails the log like tail -F
rockfish ingest -i /var/log/suricata/eve.json \
  -o /data/rockfish --sensor my-sensor --hive --follow

# From a Unix socket (Suricata unix_stream output)
rockfish ingest --socket /var/run/suricata/eve.sock \
  -o /data/rockfish --sensor my-sensor --hive

Verify Output

ls -la /data/rockfish/my-sensor/
# alert/  flow/  dns/  http/  tls/  ...

2. Run Threat Detection

# Hunt across the last 24 hours
rockfish hunt -d /data/rockfish --sensor my-sensor --hive \
  -t "24 hours"

Hunt findings are written to /data/rockfish/my-sensor/hunt/*.parquet.

View Results on Stdout

# Pretty-printed JSON
rockfish hunt -d /data/rockfish --sensor my-sensor --hive \
  --stdout --pretty

# Table format
rockfish hunt -d /data/rockfish --sensor my-sensor --hive \
  --stdout --format table

3. Generate HTML Report

# Generate report for the last 24 hours
rockfish report -d /data/rockfish --sensor my-sensor --hive \
  -t "24 hours" -o /var/www/html/ndr

Open report/index.html in a browser to view the dashboard.

Demo Mode

Generate a report with synthetic data to see all features:

rockfish report --demo -o ./demo-report

4. Publish Alerts

# Publish to MQTT broker
rockfish alert -d /data/rockfish --sensor my-sensor --hive \
  --mqtt-broker mosquitto -t "1 hour"

# Continuous publishing
rockfish alert -d /data/rockfish --sensor my-sensor --hive \
  --mqtt-broker mosquitto --continuous

Subscribe to Alerts

# In another terminal, subscribe to all rockfish topics
mosquitto_sub -t 'rockfish/#' -v

5. Continuous Operation

Run all components together for ongoing monitoring:

# Terminal 1: Continuous ingestion
rockfish ingest --socket /var/run/suricata/eve.sock \
  -o /data/rockfish --sensor prod-01 --hive

# Terminal 2: Hourly threat hunts
rockfish hunt -d /data/rockfish --sensor prod-01 --hive \
  --continuous --interval-minutes 60

# Terminal 3: Report regeneration every 5 minutes
rockfish report -d /data/rockfish --sensor prod-01 --hive \
  --continuous --interval-minutes 5

# Terminal 4: Alert publishing
rockfish alert -d /data/rockfish --sensor prod-01 --hive \
  --mqtt-broker mosquitto --continuous

Using a Configuration File

Create rockfish.yaml to avoid repeating CLI arguments:

sensor:
  name: prod-01

input:
  socket: /var/run/suricata/eve.sock

output:
  dir: /data/rockfish
  hive_partitioning: true
  compression: zstd

s3:
  bucket: rockfish-data
  region: us-east-1

alert:
  mqtt:
    broker: mosquitto
    port: 1883
    topic_prefix: rockfish

rockfish -c rockfish.yaml ingest
rockfish -c rockfish.yaml hunt --continuous
rockfish -c rockfish.yaml report --continuous
rockfish -c rockfish.yaml alert --continuous

Next Steps

• Configuration - Full YAML configuration reference
• rockfish ingest - Input sources, S3 upload, event filtering
• rockfish hunt - Detection types and tuning
• rockfish report - Report pages and theming
• rockfish alert - MQTT/Kafka publishing

Configuration

Rockfish NDR uses YAML-based configuration with CLI overrides and environment-file credential management.

Configuration Search Paths

Rockfish searches for configuration in this order:

1. --config <path> (CLI argument)
2. ./rockfish.yaml
3. /etc/rockfish/rockfish.yaml
4. ~/.config/rockfish/rockfish.yaml

Full Configuration Reference

# ============================================================
# Sensor
# ============================================================
sensor:
  name: prod-sensor-01        # Sensor name (default: hostname)

# ============================================================
# Input — EVE JSON source
# ============================================================
input:
  file: /var/log/suricata/eve.json   # Path to EVE JSON file
  socket: /var/run/suricata/eve.sock # Or: Unix socket path
  socket_type: stream                # stream (default) or dgram
  follow: true                       # Tail file like tail -F

# ============================================================
# Output — Parquet destination
# ============================================================
output:
  dir: /data/rockfish              # Output directory
  hive_partitioning: true          # year=YYYY/month=MM/day=DD/
  compression: zstd                # none, snappy, zstd
  flush_interval: 60               # Seconds between flushes
  memory_threshold: 1073741824     # 1 GB memory flush threshold
  partition: true                  # Partition by event type

# ============================================================
# Event Filtering
# ============================================================
events:
  include:                         # Only process these types
    - alert
    - flow
    - dns
    - http
    - tls
  exclude:                         # Skip these types
    - stats

# ============================================================
# S3 Upload
# ============================================================
s3:
  bucket: rockfish-data
  region: us-east-1
  prefix: ""                       # Optional key prefix
  delete_after_upload: false       # Delete local files after upload

# ============================================================
# Report
# ============================================================
report:
  output_dir: ./report
  time_window: "24 hours"
  theme: /etc/rockfish/theme.yaml  # Optional theme file
  custom_css: ""                   # Optional custom CSS path

# ============================================================
# Hunt
# ============================================================
hunt:
  time_window: "24 hours"
  detections: "beaconing,lateral,fanout,portscan,community"
  min_severity: medium
  scoring_method: hbos             # hbos or iforest
  internal_networks: "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"

# ============================================================
# Alert — MQTT and Kafka publishing
# ============================================================
alert:
  mqtt:
    broker: localhost
    port: 1883
    client_id: rockfish-alert
    qos: 1
    topic_prefix: rockfish
    username: ""
    password: ""
    tls_enabled: false
  kafka:                           # Optional — requires kafka feature
    brokers: "localhost:9092"
    topic_prefix: rockfish
    client_id: rockfish-alert
    security_protocol: plaintext
    compression: none
  confidence_threshold: 0.75
  poll_interval_secs: 30
  heartbeat_interval_secs: 60
  dedup_window_secs: 300
  enabled_types:
    - signature
    - lateral_movement
    - c2_beacon
    - exfiltration
    - anomaly

# ============================================================
# Enrichment
# ============================================================
enrichment:
  geoip:
    database_path: /usr/share/GeoIP/GeoLite2-City.mmdb
    asn_database_path: /usr/share/GeoIP/GeoLite2-ASN.mmdb
  ip_reputation:
    enabled: true
    api_key: ${ABUSEIPDB_API_KEY}
    cache_path: /var/lib/rockfish/ip_cache.parquet
    cache_ttl_hours: 48
    memory_cache_size: 50000
    lookup_timeout_ms: 200

# ============================================================
# Data Retention
# ============================================================
retention: 30d                     # 30 days (supports: 7d, 24h, etc.)

# ============================================================
# HTTP Server
# ============================================================
http:
  dir: /var/lib/report           # Directory to serve
  host: 127.0.0.1                # Bind address
  port: 8001                     # Bind port
  users_file: /opt/rockfish/etc/users  # Password file path
  session_expiry_hours: 24       # Session cookie lifetime
  auth: true                     # Enable authentication (false to disable)

# ============================================================
# License
# ============================================================
license: /etc/rockfish/license.json

Environment File

Credentials and secrets should be stored in an environment file rather than the YAML config:

# /opt/rockfish/etc/rockfish.env
ROCKFISH_S3_BUCKET=rockfish-data
ROCKFISH_S3_REGION=us-east-1
AWS_ACCESS_KEY_ID=AKIAEXAMPLE
AWS_SECRET_ACCESS_KEY=secretkey
ABUSEIPDB_API_KEY=your-api-key
MQTT_PASSWORD=broker-password
KAFKA_PASSWORD=kafka-password

The environment file path defaults to /opt/rockfish/etc/rockfish.env and can be overridden with --env-file.

CLI Overrides

CLI arguments override YAML configuration values:

# Override sensor name and data directory
rockfish -c rockfish.yaml ingest --sensor custom-name -o /tmp/data

# Override MQTT broker for alert command
rockfish -c rockfish.yaml alert --mqtt-broker custom-host

Environment Variable Overrides

Alert command options can also be set via environment variables:

Licensing

Rockfish NDR uses Ed25519-signed licenses with tier-based feature restrictions.

License Tiers

All tiers include:

• Full ingest pipeline (all event types)
• Static HTML report generation
• Demo mode with synthetic data
• MCP query server
• Alert publishing (MQTT/Kafka)
• Data retention management

License File

Licenses are JSON files with an Ed25519 signature:

{
  "id": "rockfish_acme-corp-enterprise_Abc123",
  "tier": "enterprise",
  "customer_name": "Acme Corp",
  "customer_email": "[email protected]",
  "max_flows_per_min": null,
  "issued_at": "2026-01-01T00:00:00Z",
  "expires_at": "2027-01-01T00:00:00Z",
  "signature": "base64-encoded-ed25519-signature"
}

Configuration

Specify the license file on the command line or in YAML config:

# CLI argument
rockfish --license /etc/rockfish/license.json ingest -i eve.json

# Or in rockfish.yaml
license: /etc/rockfish/license.json

Verify License

# Show license information with rockfish config
rockfish --license /etc/rockfish/license.json config

Community Tier

When no license file is provided, Rockfish operates in Community tier:

• Flow rate limited to 40,000 flows/min
• GeoIP and IP reputation enrichment disabled
• Hunt engine disabled
• All other features fully functional

Next Steps

• License Tiers - Detailed feature matrix

rockfish ingest

Ingest Suricata EVE JSON logs and write columnar Parquet files.

Overview

The ingest command reads Suricata’s EVE JSON log output — from files, stdin, or a Unix socket — partitions events by type, and writes compressed Parquet files. It supports continuous ingestion via follow mode, automatic S3 upload, and hive-style date partitioning.

Usage

rockfish ingest [OPTIONS]

Input Sources

File Input

# From a file
rockfish ingest -i /var/log/suricata/eve.json

# From stdin
cat eve.json | rockfish ingest -i -

Unix Socket

Connect directly to Suricata’s Unix socket output:

# Stream socket (default)
rockfish ingest --socket /var/run/suricata/eve.sock

# Datagram socket
rockfish ingest --socket /var/run/suricata/eve.sock --socket-type dgram

Follow Mode

Tail a live log file like tail -F, handling log rotation and truncation:

rockfish ingest -i /var/log/suricata/eve.json --follow

Follow mode saves state to a .state file so ingestion can resume after restart. Use --from-beginning to ignore saved state.

Output

Directory Layout

Flat layout (default):

output/my-sensor/
  alert/alert_2026-02-16T14-00-00.parquet
  flow/flow_2026-02-16T14-00-00.parquet
  dns/dns_2026-02-16T14-00-00.parquet

Hive-partitioned layout (--hive):

output/my-sensor/
  alert/year=2026/month=02/day=16/alert_2026-02-16T14-00-00.parquet
  flow/year=2026/month=02/day=16/flow_2026-02-16T14-00-00.parquet

Options

Event Type Filtering

# Only process alerts and flows
rockfish ingest -i eve.json --include alert,flow

# Process everything except stats
rockfish ingest -i eve.json --exclude stats

Supported types: alert, flow, dns, http, tls, fileinfo, anomaly, smtp, ssh, stats, dhcp, mqtt, modbus, dnp3, and more.

S3 Upload

Enable automatic S3 upload after each Parquet flush:

rockfish ingest -i eve.json --s3

Credentials are read from the environment file (/opt/rockfish/etc/rockfish.env):

ROCKFISH_S3_BUCKET=rockfish-data
ROCKFISH_S3_REGION=us-east-1
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...

The S3 bucket layout mirrors the local directory:

s3://bucket/{sensor}/{event_type}/year=YYYY/month=MM/day=DD/*.parquet

Examples

# Basic file ingestion with hive partitioning
rockfish ingest -i eve.json -o /data/rockfish --sensor prod-01 --hive

# Continuous socket ingestion with S3 upload
rockfish ingest --socket /var/run/suricata/eve.sock \
  -o /data/rockfish --sensor prod-01 --hive --s3

# Follow mode with custom flush interval
rockfish ingest -i /var/log/suricata/eve.json \
  --follow --flush-interval 30 --sensor edge-01

# Ingest only alerts and flows
rockfish ingest -i eve.json --include alert,flow -vv

rockfish hunt

Run graph-based behavioral threat detection on Parquet flow data.

Overview

The hunt engine builds a communication graph from network flow data and applies configurable detection algorithms to identify threats beyond signature matching — C2 beaconing, lateral movement, data exfiltration, and more.

Findings are scored with anomaly detection models (HBOS or Isolation Forest), assigned severity levels, and mapped to MITRE ATT&CK tactics.

Usage

rockfish hunt [OPTIONS]

Detection Types

Select Specific Detections

rockfish hunt -d /data --sensor my-sensor --hive \
  --detections beaconing,lateral,fanout

Output

Parquet (default)

Findings are written to {data-dir}/{sensor}/hunt/*.parquet for ingestion into the report.

Stdout

# JSON output
rockfish hunt -d /data --sensor my-sensor --stdout

# Pretty-printed JSON
rockfish hunt -d /data --sensor my-sensor --stdout --pretty

# Table format
rockfish hunt -d /data --sensor my-sensor --stdout --format table

Severity Filtering

# Only high and critical findings
rockfish hunt -d /data --sensor my-sensor --min-severity high

Tuning

Anomaly Scoring Algorithms

All detections produce findings that are scored using one of two anomaly detection methods. When a detection has 5 or more candidate groups, statistical scoring is applied; otherwise, fixed severity thresholds are used.

HBOS (Histogram-Based Outlier Scoring)

The default scoring method. HBOS builds equal-width histograms for each feature dimension, then scores each observation based on how rare its bin is.

How it works:

1. For each feature (e.g., coefficient of variation, connection count, byte ratio), divide the observed range into 10 equal-width bins
2. Count the number of observations in each bin to compute density: density = count_in_bin / total_observations
3. Apply a floor to prevent log(0): density = max(density, 0.5 / total)
4. Compute per-feature score: score = -log10(density) — rarer bins produce higher scores
5. Sum all feature scores for the final anomaly score

Properties:

• Assumes feature independence (no cross-feature correlations)
• O(n) time complexity — fast, single-pass histogram construction
• Supports feature inversion for metrics where lower values are more suspicious (e.g., beacon CV where 0.01 is more suspicious than 0.5)

Isolation Forest (iForest)

An ensemble method that isolates anomalies using random partitioning trees.

How it works:

1. Build 100 random isolation trees, each sampling 256 data points
2. Each tree recursively partitions data by randomly selecting a feature and split value until each point is isolated (or max depth is reached)
3. For each observation, compute the average path length across all trees — anomalies are isolated in fewer splits
4. Convert to anomaly score: score = 2^(-avg_path_length / c(n)) where c(n) is the expected path length for a balanced BST
5. Transform to match HBOS scale: final_score = -log10(1 - raw_score)

Properties:

• Captures cross-feature interactions (unlike HBOS)
• More robust to feature scaling
• Higher computational cost than HBOS
• Deterministic (seed = 42 for reproducibility)

Select scoring method:

rockfish hunt -d /data --sensor my-sensor --scoring-method iforest

Severity Mapping

Anomaly scores are mapped to severity levels using percentile-based thresholds across the entire finding population:

If the maximum score across all findings is below 2.0, severity is capped at Medium to suppress false positives in benign environments.

Detection Algorithm Details

Beaconing

Detects C2 callbacks by measuring the regularity of connection intervals.

1. Group connections by (src_ip, dest_ip, dest_port)
2. Compute inter-arrival time intervals between consecutive connections
3. Calculate the coefficient of variation: CV = stddev / mean
4. A perfect beacon has CV ≈ 0; random traffic has CV ≈ 1.0

Scoring features: CV (inverted), connection count, mean interval, byte consistency (CV of payload sizes)

Lateral Movement

Detects multi-hop attack chains where internal hosts are progressively compromised.

1. Build a temporal adjacency graph: src → [(dest, timestamp)]
2. Identify pivot hosts (both source and destination)
3. For each pivot, look for inbound → outbound sequences within a 1-hour window
4. Extend chains recursively (up to 10 hops)

C2 Fanout

Detects a single external IP receiving connections from many internal hosts (botnet controller pattern).

Port Scan

Detects hosts probing many ports on a target.

1. Count distinct destination ports per (src_ip, dest_ip) pair
2. Detect sequential port runs (e.g., 80-84) and compute sequential_ratio
3. Compute scan rate (ports per second)

Scoring features: unique ports, flow count, sequential ratio, scan rate

Community Detection

Identifies botnet-like clusters using Kosaraju’s Strongly Connected Components algorithm.

1. Build a directed graph from flow data
2. Find SCCs where every node can reach every other node
3. Compute density: edges / (n × (n-1))

DNS Tunneling

Detects data exfiltration encoded in DNS subdomain labels.

1. Pre-filter: average subdomain length must exceed 15 characters
2. Analyze unique subdomain count, TXT record ratio, and query rate per base domain

Scoring features: unique subdomains, avg label length, TXT ratio, query rate

Data Exfiltration

Detects internal hosts uploading disproportionate data volumes to external hosts.

1. Compute byte ratio: bytes_out / (bytes_out + bytes_in) — ratio ≥ 0.8 is suspicious
2. Filter: minimum 10 MB outbound

Scoring features: total bytes out, byte ratio, flow count

New Connection Pair

Detects (src_ip, dest_ip, dest_port) tuples never seen in the 7-day baseline window. Particularly important for OT/IoT networks where traffic is highly deterministic.

Known OT ports (Modbus 502, DNP3 20000, MQTT 1883/8883, BACnet 47808, EtherNet/IP 44818, S7comm 102, OPC UA 4840, IEC 104 2404) trigger elevated severity.

Polling Disruption

Detects when previously periodic communication becomes irregular or stops entirely. Designed for SCADA/OT environments.

1. Identify connections periodic in baseline (CV ≤ 0.3)
2. Detect disruption: either stopped (0 recent flows) or irregular (recent CV > 0.8)

Baseline Deviation

Detects significant deviations from historical traffic patterns.

1. Compare recent (1 hour) vs baseline (7 days) for same connection tuples
2. Compute ratios: flow_ratio = recent / baseline, bytes_ratio = recent / baseline
3. Flag new protocols not seen in baseline

Scoring features: flow count ratio, bytes ratio, new protocol count

Continuous Mode

# Run every hour (default)
rockfish hunt -d /data --sensor my-sensor --hive --continuous

# Run every 15 minutes
rockfish hunt -d /data --sensor my-sensor --hive \
  --continuous --interval-minutes 15

Time Window

rockfish hunt -d /data --sensor my-sensor -t "24 hours"  # default
rockfish hunt -d /data --sensor my-sensor -t "7 days"
rockfish hunt -d /data --sensor my-sensor -t "1 hour"

Examples

# Standard 24-hour threat hunt
rockfish hunt -d /data/rockfish --sensor prod-01 --hive -t "24 hours"

# Continuous with high severity filter
rockfish hunt -d /data --sensor prod-01 --hive \
  --continuous --interval-minutes 30 --min-severity high

# Beaconing with custom thresholds
rockfish hunt -d /data --sensor my-sensor \
  --detections beaconing --min-beacon-connections 50 --max-beacon-cv 0.15

rockfish report

Generate a self-contained, multi-page HTML NDR report from Parquet data.

Overview

The report command produces interactive HTML dashboards with Chart.js and D3.js visualizations — no web server required. Reports include 12+ pages covering alerts, threats, DNS, TLS, flows, hosts, network topology, asset inventory, and hunt findings.

Usage

rockfish report [OPTIONS]

Report Pages

Visualization Features

• World Map — Leaflet.js with country-level flow, alert, and reputation overlays
• Network Graph — D3.js force-directed topology with Flows/Alerts/Hunt toggle layers, including anomaly (iForest/HBOS) findings overlay
• Heat-Mapped Tables — Gradient backgrounds for volume, severity, and scores
• Collapsible Tables — Expand/collapse with JSON export
• Severity Colors — Consistent palette: critical (red) through info (blue)

Options

Theming

Customize report appearance with a YAML theme file:

# theme.yaml
background: "#0d1117"
surface: "#161b22"
text: "#e6edf3"
text_heading: "#ffffff"
accent: "#1a73e8"

rockfish report -d /data --sensor my-sensor --theme theme.yaml

See theme.yaml.example for all available options.

Custom Logo

Replace the default Rockfish logo with your own branding. Requires Standard or Enterprise license.

# theme.yaml
logo_path: "/path/to/your-logo.png"

The logo appears in the header bar of every report page.

Demo Mode

Generate a report with synthetic data to showcase all features:

rockfish report --demo -o ./demo-report

Demo mode is available on all license tiers.

Continuous Mode

# Regenerate every 5 minutes (default)
rockfish report -d /data --sensor my-sensor --hive --continuous

# Regenerate every 15 minutes
rockfish report -d /data --sensor my-sensor --hive \
  --continuous --interval-minutes 15

Examples

# 24-hour report
rockfish report -d /data/rockfish --sensor prod-01 --hive \
  -o /var/www/html/ndr

# 7-day report with custom theme
rockfish report -d /data --sensor prod-01 --hive \
  -t "7 days" --theme /etc/rockfish/theme.yaml

# Continuous regeneration for live dashboard
rockfish report -d /data --sensor prod-01 --hive \
  --continuous --interval-minutes 10 -o /var/www/html/ndr

rockfish alert

Publish detection events to MQTT and/or Apache Kafka for automated response.

Overview

The alert command reads Suricata alert and hunt finding Parquet data, normalizes events into a common JSON payload, and publishes them to MQTT and/or Kafka topics. It supports deduplication, rate limiting, confidence filtering, and continuous polling.

This is the “R” (Response) in NDR — enabling closed-loop automated response via n8n, Node-RED, or any MQTT/Kafka consumer.

Usage

rockfish alert [OPTIONS]

Alert Payload

All alerts are normalized to a common JSON schema:

{
  "alert_id": "RF-2026-00042",
  "timestamp": "2026-02-16T14:32:07Z",
  "detection_type": "signature",
  "confidence": 0.95,
  "source": {
    "ip": "10.0.12.45"
  },
  "destinations": [
    { "ip": "185.220.101.34", "port": 443 }
  ],
  "metadata": {
    "protocol": "TCP",
    "suricata_sid": 2025001,
    "suricata_signature": "ET MALWARE Cobalt Strike Beacon",
    "suricata_category": "A Network Trojan was detected",
    "community_id": "1:abc123"
  },
  "recommended_action": "block_ip"
}

Topic Mapping

MQTT Topics (forward slashes)

Kafka Topics (dots)

MQTT Options

Kafka Options

Requires building with the kafka feature.

Supported security protocols: plaintext, ssl, sasl_plaintext, sasl_ssl

Supported compression: none, gzip, snappy, lz4, zstd

Confidence Mapping

Suricata severity:

Hunt severity:

Deduplication

Identical alerts (same source IP, destination IP, detection type, and SID) are suppressed within a configurable time window (default: 5 minutes).

YAML Configuration

alert:
  mqtt:
    broker: mosquitto
    port: 1883
    client_id: rockfish-alert
    qos: 1
    topic_prefix: rockfish
  kafka:
    brokers: "kafka1:9092,kafka2:9092"
    topic_prefix: rockfish
    security_protocol: sasl_ssl
    compression: snappy
  confidence_threshold: 0.75
  poll_interval_secs: 30
  dedup_window_secs: 300
  enabled_types:
    - signature
    - lateral_movement
    - c2_beacon

Heartbeat

Periodic heartbeat published to {prefix}/status/heartbeat:

{
  "timestamp": "2026-02-16T14:32:07Z",
  "uptime_secs": 3600,
  "alerts_published": 142,
  "status": "running"
}

Examples

# Single-shot MQTT publish
rockfish alert -d /data --sensor my-sensor --hive \
  --mqtt-broker mosquitto -t "1 hour"

# Continuous MQTT + Kafka publishing
rockfish alert -d /data --sensor my-sensor --hive \
  --mqtt-broker mosquitto \
  --kafka --kafka-brokers kafka1:9092,kafka2:9092 \
  --continuous

# High-confidence only with TLS
rockfish alert -d /data --sensor prod-01 --hive \
  --mqtt-broker mqtt.internal --mqtt-tls \
  --confidence-threshold 0.85 --continuous

n8n Integration

Subscribe to MQTT topics in n8n for automated response:

1. Add an MQTT Trigger node subscribing to rockfish/alerts/#
2. Parse the alert JSON payload
3. Route by detection_type
4. Execute response actions (block IP, quarantine host, create ticket)

rockfish mcp

Start an MCP (Model Context Protocol) server for AI-powered queries on Parquet data.

Overview

The MCP server exposes Parquet data to AI assistants and LLM toolchains using the Model Context Protocol. It provides query tools for data exploration and hunt tools for threat detection.

Usage

rockfish mcp [OPTIONS]

Transport Modes

# stdio mode (default)
rockfish mcp

# HTTP mode
rockfish mcp -t http --host 0.0.0.0 --port 3000

Built-in Tools

Query Tools

Hunt Tools

Options

Authentication

HTTP mode supports JWT token authentication and OAuth2. See the MCP authentication documentation for configuration details.

Examples

# Start MCP server for Claude Desktop
rockfish mcp --data-dir /data/rockfish --sensor prod-01

# HTTP mode for web clients
rockfish mcp -t http --host 0.0.0.0 --port 3000 \
  --data-dir /data/rockfish --sensor prod-01

rockfish chat

Start an AI-powered chat server for conversational network security analysis.

Overview

The chat server provides a web-based conversational interface for querying network security data using natural language. It integrates with MCP for live data queries and supports pluggable LLM backends.

Usage

rockfish chat [OPTIONS]

LLM Modes

# Local SLM mode
rockfish chat --mode slm

# Cloud mode
rockfish chat --mode cloud

# Hybrid mode
rockfish chat --mode hybrid

Data Modes

Options

Features

• Session management with state persistence
• Security guardrails and response caching
• MCP integration for live data queries
• Natural language to SQL translation

Examples

# Start chat server with local SLM
rockfish chat --mode slm \
  --data-dir /data/rockfish --sensor prod-01

# Start with MCP integration
rockfish chat --mode cloud \
  --mcp-endpoint http://localhost:3000 \
  --host 0.0.0.0 --port 8082

rockfish http

Serve static report files over HTTP with optional session-based authentication.

Overview

The http command runs a lightweight HTTP server to serve generated report pages behind a login wall. Designed to sit behind a reverse proxy (nginx, Caddy, etc.), it provides user management, session cookies, and static file serving with automatic index.html resolution.

Usage

rockfish http [OPTIONS] [COMMAND]

Options

User Management

Manage users with the user subcommand:

# Add a new user (prompts for password)
rockfish http user add <username>

# Delete an existing user
rockfish http user del <username>

# List all configured users
rockfish http user list

Password File

Users are stored in a password file (default: /opt/rockfish/etc/users):

# Rockfish HTTP users
# Format: username:sha256hex
admin:e3b0c44298fc1c149afb...
analyst:d7a8fbb307d7809469...

• Passwords are SHA-256 hashed before storage
• File permissions are set to 0600 (owner read/write only)
• The file is reloaded on each login attempt (no restart needed to add users)

Authentication

When authentication is enabled (default), the server:

1. Redirects unauthenticated requests to /login
2. Accepts username/password via HTML login form
3. Validates credentials using constant-time comparison (subtle crate)
4. Issues a session cookie (rockfish_session) on successful login
5. Expired sessions are cleaned up automatically every 5 minutes

Session Cookies

Audit Logging

All login attempts are logged to stderr:

LOGIN OK   user=admin from=192.168.1.100
LOGIN FAIL user=unknown from=10.0.0.5

Configuration

The HTTP server can be configured via YAML:

http:
  dir: /var/lib/report
  host: 127.0.0.1
  port: 8001
  users_file: /opt/rockfish/etc/users
  session_expiry_hours: 24
  auth: true

CLI arguments override YAML values.

Examples

# Serve reports with authentication
rockfish http --dir /var/www/html/ndr

# Serve on all interfaces (e.g., behind reverse proxy)
rockfish http --dir /var/lib/report --host 0.0.0.0 --port 8080

# Serve without authentication (local/demo use)
rockfish http --dir ./report --no-auth

# Create admin user, then start server
rockfish http user add admin
rockfish http --dir /var/lib/report

Reverse Proxy (nginx)

server {
    listen 443 ssl;
    server_name ndr.example.com;

    ssl_certificate     /etc/ssl/certs/ndr.pem;
    ssl_certificate_key /etc/ssl/private/ndr.key;

    location / {
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Systemd Service

[Unit]
Description=Rockfish NDR Report Server
After=network.target

[Service]
ExecStart=/opt/rockfish/bin/rockfish http --dir /var/lib/report --host 127.0.0.1 --port 8001
Restart=on-failure
User=rockfish

[Install]
WantedBy=multi-user.target

Continuous Report + HTTP Server

Run report generation and the HTTP server together:

# Terminal 1: Regenerate reports every 10 minutes
rockfish report -d /data --sensor prod-01 --hive \
  --continuous --interval-minutes 10 -o /var/lib/report

# Terminal 2: Serve reports with authentication
rockfish http --dir /var/lib/report

rockfish prune

Remove old Parquet files based on a retention policy.

Overview

The prune command enforces data retention by removing Parquet files older than a configurable window. Supports dry-run preview and per-event-type granularity.

Usage

rockfish prune [OPTIONS]

Options

Retention Periods

Supports various time formats:

rockfish prune -r "30d"        # 30 days
rockfish prune -r "7d"         # 7 days
rockfish prune -r "24h"        # 24 hours
rockfish prune -r "30 days"    # 30 days (verbose)

Examples

# Preview what would be deleted (dry run)
rockfish prune -d /data/rockfish --sensor prod-01 --hive \
  -r "30d" --dry-run

# Delete files older than 7 days
rockfish prune -d /data/rockfish --sensor prod-01 --hive -r "7d"

# Prune only alert and flow data
rockfish prune -d /data --sensor prod-01 \
  -r "14d" --event-types alert,flow

# Prune with inventory files
rockfish prune -d /data --sensor prod-01 \
  -r "30d" --include-inventory

rockfish config

Show resolved configuration, enabled features, and system information.

Overview

The config command displays the current configuration including YAML settings, environment file values (with secrets masked), enabled compile-time features, and license information.

Usage

rockfish config [OPTIONS]

Output

The command displays:

• Config file — resolved path to the active YAML configuration
• Environment file — loaded environment variables (secrets masked as XXX...last8)
• Enabled features — compile-time feature flags
• License information — tier, customer, expiration (if a license file is provided)

Examples

# Show all configuration
rockfish config

# With a specific config file
rockfish -c /etc/rockfish/rockfish.yaml config

# With license info
rockfish --license /etc/rockfish/license.json config

Feature Flags

The config output includes which features were compiled in:

Compile-time Features
---------------------
  s3           enabled
  mcp          enabled
  hunt         enabled
  report       enabled
  chat         enabled
  alert        enabled
  kafka        disabled
  geoip        enabled
  ip_reputation enabled

rockfish stats

Parse EVE JSON and show statistics without writing output (dry run).

Overview

The stats command reads Suricata EVE JSON and displays event type counts and basic statistics. Useful for inspecting log files before ingestion.

Usage

rockfish stats -i <INPUT> [OPTIONS]

Options

Examples

# Show event type distribution
rockfish stats -i /var/log/suricata/eve.json

# Show first 10 events
rockfish stats -i eve.json --show 10

# From stdin
cat eve.json | rockfish stats -i -

Enrichment

Rockfish NDR enriches flow data with geographic and reputation intelligence during ingestion.

GeoIP

Geographic location lookups via MaxMind GeoLite2 or GeoIP2 databases.

Enriched Fields

Configuration

enrichment:
  geoip:
    database_path: /usr/share/GeoIP/GeoLite2-City.mmdb
    asn_database_path: /usr/share/GeoIP/GeoLite2-ASN.mmdb

Requirements

• MaxMind GeoLite2-City and GeoLite2-ASN databases
• Free account at maxmind.com
• Requires the geoip feature (enabled by default)
• Requires Standard or Enterprise license tier

IP Reputation

Abuse confidence scoring via the AbuseIPDB API.

Enriched Fields

Configuration

enrichment:
  ip_reputation:
    enabled: true
    api_key: ${ABUSEIPDB_API_KEY}
    cache_path: /var/lib/rockfish/ip_cache.parquet
    cache_ttl_hours: 48
    memory_cache_size: 50000
    lookup_timeout_ms: 200
    fail_open: false

Caching

IP reputation lookups are cached at two levels:

1. Memory cache — LRU cache (default: 50,000 entries) for fast lookups
2. Parquet cache — Persistent disk cache with configurable TTL

Requirements

• AbuseIPDB API key (set in environment file)
• Requires the ip_reputation feature (enabled by default)
• Requires Standard or Enterprise license tier

Report Integration

Both GeoIP and IP reputation data appear across multiple report pages:

• Overview — Top countries by flow volume
• Flows — Country breakdown with GeoIP data
• Threats — IP reputation scores and flagged hosts
• Network — Node detail panel with geographic info
• World Map — Leaflet.js globe with country-level overlays

Note: GeoIP and IP reputation columns are only populated when the probe runs with those features enabled. Report queries gracefully return zero rows when enrichment data is absent.

Asset Inventory

Passive device discovery from observed network traffic.

Overview

Rockfish builds an asset inventory by analyzing network flow patterns, extracting DHCP metadata, and inferring device roles — all without agents or active scanning.

Capabilities

Inferred Device Roles

OT Protocol Support

Report Integration

The Inventory report page displays:

• Device list with inferred roles and protocol usage
• New/unknown device alerts
• OT protocol traffic summary
• First-seen and last-seen timestamps
• Communication pattern metrics (connection count, bytes)

Deployment Profiles

Rockfish NDR supports different deployment profiles optimized for specific environments.

IT Profile

Enterprise network monitoring with full enrichment capabilities.

# IT deployment example
enrichment:
  geoip:
    database_path: /usr/share/GeoIP/GeoLite2-City.mmdb
  ip_reputation:
    enabled: true
    api_key: ${ABUSEIPDB_API_KEY}
s3:
  bucket: security-data
  region: us-east-1
alert:
  mqtt:
    broker: siem-mqtt.internal

OT Profile

Industrial / IoT environments with asset inventory and baseline monitoring.

# OT deployment example
hunt:
  detections: "baseline_deviation,polling_disruption,beaconing,lateral"
  internal_networks: "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"

Key OT detections:

• Baseline deviation — traffic pattern shifts in control networks
• Polling disruption — interruption of periodic SCADA polling
• New connection pairs — unexpected host communication

Military Profile

Air-gapped networks with no internet dependencies.

# Air-gapped deployment example
enrichment:
  geoip:
    database_path: /opt/rockfish/geoip/GeoLite2-City.mmdb
  ip_reputation:
    enabled: false
hunt:
  detections: "exfiltration,beaconing,lateral,fanout,dns_tunneling"
alert:
  mqtt:
    broker: localhost

Key considerations:

• All enrichment data must be pre-loaded locally
• SLM (small language model) runs entirely on-device
• No S3, no cloud APIs, no external network access
• Focus on data exfiltration and unauthorized communication detection

Docker Deployment

Rockfish NDR provides Docker images for containerized deployment.

Production Container

# Build
docker build -t rockfish:latest -f Dockerfile .

# Run with mounted config and data
docker run -d --name rockfish \
  -v /opt/rockfish/etc:/opt/rockfish/etc:ro \
  -v /data/rockfish:/data/rockfish \
  -p 3000:3000 -p 8082:8082 \
  rockfish:latest ingest --socket /var/run/suricata/eve.sock

The production image is a multi-stage build (Rust builder, Debian slim runtime) and includes all default features plus Kafka, with DuckDB bundled from source.

Demo Container

# Build and run demo report on port 8080
docker build -t rockfish-demo:latest -f Dockerfile.demo .
docker run -d --name rockfish-demo -p 8080:8080 rockfish-demo:latest

The demo image generates a synthetic report at build time and serves it via nginx.

License Tiers

Detailed feature matrix for Rockfish NDR license tiers.

Tier Comparison

Community Tier

Available without a license file. Provides core NDR functionality:

• Ingest all Suricata event types to Parquet
• Generate HTML reports (without enrichment overlays)
• MCP server for AI-powered queries
• Alert publishing to MQTT/Kafka
• Demo mode with synthetic data
• Data retention management

Limitation: 40,000 flows per minute.

Standard Tier

Adds enrichment intelligence:

• Everything in Community
• MaxMind GeoIP lookups (country, city, ASN)
• AbuseIPDB IP reputation scoring
• Geographic overlays in reports
• Reputation-based threat flagging

Limit: 100,000 flows per minute.

Enterprise Tier

Full NDR capability:

• Everything in Standard
• Graph-based behavioral threat hunting
• HBOS and Isolation Forest anomaly scoring
• Hunt findings integrated into reports
• 10 detection types (beaconing, lateral movement, C2 fanout, etc.)
• Continuous hunt scheduling

No flow rate limit.

License File Format

{
  "id": "rockfish_acme-corp-enterprise_Abc123",
  "tier": "enterprise",
  "customer_name": "Acme Corp",
  "customer_email": "[email protected]",
  "max_flows_per_min": null,
  "issued_at": "2026-01-01T00:00:00Z",
  "expires_at": "2027-01-01T00:00:00Z",
  "signature": "base64-encoded-ed25519-signature"
}

Licenses are verified using Ed25519 digital signatures with a public key embedded in the binary.