Independent Review · 2 of 2 · PhD Validation
Independent PhD Audit
A deliberately stricter, independent replication and robustness audit of all four RegimeSignal™ S&P 500 prediction signals — designed not to duplicate the validation. Reviewer: Dr J. Hossain · July 2026.
Determination
VALIDATED WITH QUALIFICATIONS
All 35 locked package files were checksum-verified and reproduced. All four signals reproduced with no divergence (MBS T1 and MBS T2 bit-exact). No feature-level label leakage was found, and every signal cleared a permutation-null check.
The numbers (locked, independently reproduced)
| Signal | Precision | FPR | AUC | OOS (events) | Reproduction |
|---|---|---|---|---|---|
| BRS — Bear regime (−20%) | 85.7% | 2.5% | 0.91 | 304 (67) | Exact |
| MBS T1 — Pullback (−5%) | 83.3% | 6.5% | 0.88 | 154 (46) | Bit-exact |
| MBS T2 — Correction (−10%) | 84.4% | 4.1% | 0.92 | 154 (33) | Bit-exact |
| RRS — Recovery (+10%) | 81.8%* | 1.5% | 0.76 | 154 (23) | Exact |
| Average | ~84% | ~4% | — | ~4-mo window | 4 / 4 reproduced |
~84% = simple mean of the four locked precisions. BRS, MBS T1 and MBS T2 are audit-clean (reproduced exactly / bit-exact). *RRS 81.8% is precision-first and exploratory — see note below.
What the audit confirmed
- All 35 locked package files were checksum-verified and matched; all four signals reproduced with no divergence.
- MBS T1 and MBS T2 reproduced bit-exact (~1×10⁻⁸; every one of 154 fire/no-fire decisions in agreement).
- Walk-forward construction is sound — the predicted month is excluded from its own training window and features are lagged — with no feature-level label leakage found.
- Every signal cleared a permutation-null check — observed accuracy sits far above the label-shuffled baseline, indicating genuine signal.
- The deployed engines were benchmarked against three independent alternative learners on identical data, all within a tight AUC band — the signal is in the factor set, not the algorithm.
- The Market Break signals degraded gracefully under a 12-month forward shift of the out-of-sample window.
⚠ Qualifications — Disclosed in Full
Event counts are small across all signals, so confidence bands are wide and robustness checks are indicative rather than definitive.
The Regime Recovery Signal (RRS) is precision-first and exploratory. Its high-conviction threshold was selected on the same window it is measured on, so its 82% precision is a historical upper bound; a prospective, held-out estimate is nearer 69%. Both are reported.*
The walk-forward loop applies no purge/embargo for the four-month label horizon; BRS reproduced on a locked Layer-1 composite consumed as-is.
Area determinations
*Pass with qualifications.
*RRS operating point
82% = 9 of 11 fires correct at the locked high-conviction threshold, reproduced exactly. Because the threshold was selected on the reported out-of-sample window, this figure is in-sample-optimistic; a prospective (held-out) estimate is near 69% (Wilson 95% CI ≈ [0.52, 0.95]). Recall is ~39% by design — the recovery signal stays silent through fake-out rallies and speaks only on high-conviction turns; presented as exploratory pending out-of-sample data postdating the lock.
Two independent reviews
This audit complements our first independent review.
Determination is "Validated with Qualifications," not an unconditional validation. Full signed report available to qualified reviewers under NDA. Precision figures reflect the historical out-of-sample record and do not guarantee future results. This audit does not validate live or production deployment and does not constitute investment advice.