How we measure accuracy
Model: v5.7-ensemble · Accuracy data: May 2026
Every signal WatchSignal shows is a probability — not a guarantee. This page explains how we measure that probability, what the headline numbers mean, and what we deliberately do not claim.
Manifest accuracies (v5.7 production)
These are the published walk-forward validation accuracies for each prediction horizon in the live ensemble. Production telemetry must hold within a 5-percentage-point tolerance of these numbers — if it drifts, we know.
| Horizon | Accuracy | What it predicts |
|---|---|---|
| Short | 59.49% | Next-day directional move |
| Medium | 47.41% | ~5-day directional move |
| Long | 44.88% | ~21-day directional move |
| Macro | 47.11% | Regime / macro-context head |
Numbers are F1 scores from the v5.7 retrain (70-feature trio + 121-feature macro head). Source of truth is MANIFEST_ACCURACY in the app's ModelPerformanceService — this page updates in lockstep with the manifest.
For context: a coin flip is 50% on a two-class problem and ~33% on the three-class (Bullish / Neutral / Bearish) problem the model actually solves. The short head's 59.49% reflects that three-class setting, not a two-class accuracy.
What confidence means
Every signal carries a calibrated confidence percentage. When the app shows 70%, roughly 70 of 100 such signals were correct on our walk-forward validation set.
That's the goal of calibration: the number on the screen should match the empirical hit rate. We monitor calibration in production — if the model's 70%-confidence signals start hitting at 55%, we ship a recalibration.
A Bullish signal at 88% confidence is very different from one at 55%. Always factor confidence into your interpretation.
How we validate
Walk-forward cross-validation. We split the historical price record into rolling windows: train on the past, predict the next window, advance, repeat. No future data ever leaks into a prediction it shouldn't see.
Purge and embargo gaps. Between each train window and its validation window we insert a gap equal to the prediction horizon plus 21 trading days. This kills auto-correlation leakage — a known failure mode in financial backtests.
No look-ahead. Features available at time t are computed using only data observable at time t. Earnings dates, macro releases, and corporate actions are aligned to their announcement timestamps, not their as-of dates.
Validation horizon. The reported accuracies are out-of-sample on data the model never saw during training.
What we don't claim
- Backtested accuracy is not live trading performance. Slippage, fees, fills, and your own decision latency are not in these numbers.
- The model is regime-sensitive. Walk-forward windows span multiple market regimes, but the next regime may not look like the last. Accuracy in a 2022-style drawdown will not match accuracy in a 2024-style melt-up.
- We don't claim profit. Directional accuracy ≠ profitability. A 60%-accurate model with a bad risk/reward ratio still loses money. Position sizing, stop-loss discipline, and trade selection are yours.
- Per-ticker accuracy varies. The manifest numbers are aggregate. Some tickers are inherently easier for the model than others — the in-app Forecast Accuracy tab on each ticker shows that ticker's actual hit rate.
- No advice, no recommendation. WatchSignal is an analytical tool. We don't tell you what to buy.
How to read a signal
- Direction — Bullish / Neutral / Bearish
- Confidence — how concentrated the model's probability is
- Horizon — which head produced it (short / medium / long / macro)
- Per-ticker accuracy — open the Forecast Accuracy tab on the ticker to see the model's track record on that name specifically
Take all four together. A 55% Bullish signal on a ticker where the model historically hits 70% is meaningfully different from a 55% Bullish signal on a ticker where it hits 48%.
WatchSignal is not a registered investment adviser or broker-dealer. AI-generated signals are informational only and do not constitute financial advice. Past performance does not guarantee future results. See our full methodology for model architecture and feature details.