ML-Based Analytics
Rockfish NDR ships four detection mechanisms that all fall under the ML-based analytics umbrella: iForest, HBOS, OCCAM, and OCCAM. Each catches threats the others miss, and the real value comes from running them together as a layered pipeline.
This page covers what each one is, how they layer, and where each shows up in the report.
The four mechanisms at a glance
| iForest | HBOS | OCCAM | OCCAM | |
|---|---|---|---|---|
| Kind | ML algorithm | Statistical algorithm | Detection engine | Detection engine |
| Built on | — | — | Uses HBOS | Uses OCCAM tokens |
| Granularity | Per flow | Per asset / window | Per asset / 15-min window | Per token observation |
| Output | Anomaly score (0..1) | Outlier score (sum of −log P(feature)) | Token + ATT&CK tactic + surprisal in bits | Disposition (suppressed / investigate / present / elevated) + Viterbi path |
| Trained | Offline (historical data) | Online (Welford streaming) | Online (HBOS internally) | Online (HMM over token vocabulary) |
| Where it shows up | Anomalies page | Reliability page | OCCAM page | Occam page |
iForest and HBOS are math — algorithms that compute anomaly scores from a feature vector. OCCAM and OCCAM are engines — pipelines that use those algorithms and add structure (per-asset windows, ATT&CK mapping, sequence modeling) to produce security-meaningful output.
Why all four ship together — the layering effect
Four engines. One layered prediction.
Each stage answers a question the layer above couldn't. Data flows left to right — from a single flow's anomaly score to a predictive alert about an in-progress attack.
Reading the diagram
| Stage | Question it answers | Output | Where it surfaces |
|---|---|---|---|
| iForest | Is THIS flow anomalous? | per-flow score | Anomalies page |
| HBOS | Did this HOST drift from itself? | drift events, SLA breaches | Reliability page |
| OCCAM | Is the drift security-meaningful? | ATT&CK-mapped tokens | OCCAM page |
| OCCAM | Does the sequence match an attack? | dispositions, pre-intrusion alerts | Occam page |
Each layer answers a question the layer above can’t:
- iForest catches the first-pass per-flow weirdness you’d otherwise miss entirely (encrypted threats, novel tooling, low-frequency C2).
- HBOS turns that weirdness into per-host context — “this happened repeatedly for asset X” — and additionally catches gradual host degradation (network reliability) for free.
- OCCAM turns that context into a security narrative mapped to the MITRE ATT&CK matrix — “X is showing exfiltration-tactic behavior” — so analysts can triage by attacker intent, not by mystery score.
- OCCAM turns the narrative into a predictive alert by recognizing multi-stage attack paths — “X is mid-attack-path; investigate now” — and suppresses isolated tokens that don’t match any known sequence.
Skipping a layer leaves a gap:
| Skip | Cost |
|---|---|
| Skip iForest | Blind to flows your IDS rules don’t already know about |
| Skip HBOS | No per-host context; performance and security teams duplicate work |
| Skip OCCAM | Analysts stuck interpreting raw anomaly scores with no security framing |
| Skip OCCAM | One-shot anomalies overwhelm the SOC; real attacks reach completion before response |
What each one is
iForest — Isolation Forest
An unsupervised ML algorithm that builds many random trees that partition the feature space; anomalies are easier to “isolate” and produce shorter average path lengths in the trees.
- Pre-trained model — Rockfish ships baseline iForest models trained on representative traffic, and supports retraining against your own data.
- Strengths: handles high-dimensional input, no distribution assumptions, works on encrypted traffic where signatures don’t.
- Weaknesses: opaque (“why is this anomalous?”), needs periodic retraining as your network evolves.
- In the report: the Anomalies page surfaces top per-flow scores.
HBOS — Histogram-Based Outlier Score
A 2012 statistical anomaly score (Goldstein, Dengel). Per-feature
histograms give the probability of seeing each value; the outlier score is
the sum of −log P(feature_i) across features. High score = the
combination of feature values is rare for this asset.
Rockfish runs HBOS-style baselines online using Welford’s algorithm so each asset has its own per-feature mean / variance estimate updated incrementally.
- Strengths: fast, online (no training data needed), interpretable per-feature, runs cheaply on every asset every 15 minutes.
- Weaknesses: assumes feature independence; less powerful than tree-based methods on highly correlated features.
- In the report: the Reliability page surfaces drift events and SLA breaches; HBOS also feeds the OCCAM classifier.
OCCAM — Behavioral tokenizer
Naming caveat: Rockfish OCCAM is not the OCCAM-rules format from sigmahq.io. They share a name but are unrelated. Rockfish OCCAM is a behavioral tokenizer; the public OCCAM is a YAML detection-rule grammar.
Every 15 minutes, for each asset, OCCAM aggregates window stats and
emits tokens — short labels representing observed behavior categories
(e.g. encrypted-ratio-high, slow-handshake, outbound-spike,
unusual-port-mix). Each token carries:
-
A surprisal score in bits — how rare this token is for this asset, computed via HBOS on the underlying features.
-
An ATT&CK tactic mapping — initial access, execution, lateral movement, exfiltration, etc. — so the token is security-meaningful.
-
Strengths: translates raw statistics into a vocabulary an analyst recognizes; per-asset baselining makes “rare for this host” the standard.
-
Weaknesses: single-window view; can’t see “lead-up” patterns by itself.
-
In the report: the OCCAM page surfaces tokens, dispositions, and the per-tactic breakdown.
OCCAM — Sequence predictor
Named for Occam’s razor: when multiple attack paths could explain the observed token sequence, pick the simplest. OCCAM is a Hidden Markov Model trained on historical token sequences leading up to confirmed incidents. It runs the Viterbi algorithm over the live OCCAM token stream to score how strongly the recent sequence resembles a known attack path.
Output is a per-token disposition:
| Disposition | Meaning |
|---|---|
suppressed | Isolated anomaly, no matching sequence — likely false positive |
investigate | Anomaly worth looking at but not part of a known path |
present | Token participates in a partial attack path |
elevated | Token completes or extends a recognized attack path — pre-intrusion alert |
- Strengths: forecasts attacks while they’re still in progress; cuts false positives via sequence context; explainable Viterbi paths.
- Weaknesses: depends on OCCAM tokens flowing; HMM trained on labeled attack paths.
- In the report: the Occam page surfaces dispositions, Viterbi paths, and pre-intrusion alerts.
Concrete example — the same incident through four lenses
A workstation slowly exfiltrating encrypted data over hours:
| Layer | What it produces | How an analyst reads it |
|---|---|---|
| iForest | Flow 10.0.12.45 → 198.51.100.47:443 scored 0.87 | “This single flow’s overall feature pattern resembles known-bad more than known-good.” |
| HBOS | Asset 10.0.12.45 window 14:00–14:15 has tcp_handshake_rtt_ms_avg 4.2σ above its baseline | “This host’s measurements drifted from its own history.” |
| OCCAM | Same window emits token encrypted-ratio-high · tactic=exfiltration · surprisal=8.7 bits | “The drift is behaviorally meaningful and maps to the exfiltration tactic.” |
| OCCAM | Disposition elevated; Viterbi path = recon → discovery → encrypted-ratio-high | “And the recent token sequence matches the exfiltration HMM. Investigate now.” |
Each layer answers a question the layer above couldn’t:
| Layer | Answers |
|---|---|
| iForest | Is this single flow anomalous? |
| HBOS | Did this host’s measurements deviate from itself? |
| OCCAM | Is the deviation behaviorally meaningful? |
| OCCAM | Does the recent sequence look like a known attack path? |
When to look at each
| Question / situation | Use |
|---|---|
| Score one flow in isolation | iForest |
| Detect host degradation or compromise at the network-quality level | HBOS via Reliability page |
| Triage anomalies by ATT&CK tactic | OCCAM |
| Forecast an incident before it completes | OCCAM |
| Suppress noisy false positives | OCCAM |
License tier
The Anomaly lens (iForest / HBOS) and the Behavioral lens (OCCAM HMM sequence predictor) are Professional-tier features — both gate at Professional+ and on the corresponding parquet datasets being present. The detection swimlane + topology graph that overlay these lenses are Enterprise-tier. See License Tiers for the full feature matrix.