# ACP Anomaly Detection — Benchmark Results

> ⚠️  **Disclaimer**: Results are probabilistic indicators only. Human review required before automated action.

**Run date:** 2026-04-14T20:24:25Z  
**Scorer:** `neural`  
**Seed:** 42  
**Total samples:** 200 (100 normal + 100 anomalous)  

---

## Aggregate Metrics

| Metric | Threshold 0.5 | Threshold 0.7 |
|--------|:-------------:|:-------------:|
| **Precision** | 0.9434 | 0.9804 |
| **Recall**    | 1.0000 | 1.0000 |
| **F1 Score**  | 0.9709 | 0.9901 |
| **Accuracy**  | 0.9700 | 0.9900 |
| TP / FP / FN / TN | 100/6/0/94 | 100/2/0/98 |

**ROC-AUC: 1.0000**

---

## Normal BPB Distribution (baseline)

| Stat | Value |
|------|-------|
| Mean ± Std | 1.243 ± 0.318 bpb |
| p50  | 1.187 bpb |
| p90  | 1.684 bpb |
| p95 (alert threshold) | 1.744 bpb |
| p99  | 2.051 bpb |

---

## Detection Rate by Category

| Category | n | Mean BPB | Mean Score | Detected @0.5 | Rate @0.5 | Detected @0.7 | Rate @0.7 |
|----------|:-:|:--------:|:----------:|:-------------:|:---------:|:-------------:|:---------:|
| `normal` | 100 | 1.243 | 0.142 | 6/100 `█░░░░░░░░░░░░░░░░░░░` | 6.00% | 2/100 | 2.00% |
| `random_bytes` | 25 | 7.313 | 1.000 | 25/25 `████████████████████` | 100.00% | 25/25 | 100.00% |
| `code` | 25 | 3.020 | 0.975 | 25/25 `████████████████████` | 100.00% | 25/25 | 100.00% |
| `foreign_language` | 25 | 3.287 | 0.955 | 25/25 `████████████████████` | 100.00% | 25/25 | 100.00% |
| `json` | 25 | 3.398 | 1.000 | 25/25 `████████████████████` | 100.00% | 25/25 | 100.00% |

---

## Scorer Notes

This benchmark used the **neural** scorer:

- Full ACP v4 neural BPB scorer loaded from checkpoint.
- Neural scorer assigns low BPB to in-distribution English text it was trained on.
- Out-of-distribution inputs (random bytes, unseen languages) receive high BPB.
- Model: `models/v4/micro_fineweb_ema.pt`

---

## How to Reproduce

```bash
# From workspace root:
python business_ecosystem/02_anomaly_detection/benchmarks/run_benchmark.py

# With trained neural model:
python business_ecosystem/02_anomaly_detection/benchmarks/run_benchmark.py \
    --model models/v4/micro_fineweb_ema.pt
```

*Results generated in 5.34s (19.7 ms/sample)*