Benchmarking

Model vs Market

Walk-forward predictions benchmarked against Pinnacle closing lines across Feb '26 – Apr '26. This view reads like a duel: where the model agrees, where it drifts, and where the market still wins.

410 matched games34% coverageDashboard accuracy 62.8%Report todayPinnacle closing lines

Current dashboard accuracy62.8%1,291 games · Postseason-aware GBM

Canonical site metric

The homepage and this page use the same active evaluation artifact for the headline accuracy. Market gauges below are a separate odds-matched subset and are labeled that way.

backtest_unified_playoff_gbm.csv

Oct '25 – May '26Updated today

Head-to-head — odds-matched sample

67.3%

GBM matched accuracy410 odds-matched games

77.9%

Vegas accuracyPinnacle closing

Model Brier0.205lower is better

Vegas Brier0.161Pinnacle closing

Duel read

10.6% advantage to the market

This is not the site-wide dashboard accuracy. It only covers games that could be joined to closing-line odds, so it can legitimately differ from the headline walk-forward metric.

Agreement rate

77.3%

Coverage

34%

Edge threshold analysis

Simulated flat-bet ROI at increasing edge thresholds. Negative ROI reflects market vig.

Flat $100 bets at Pinnacle closing moneyline odds. Negative ROI reflects market vig on 410 matched games.

Regular-season benchmark models — 1,214 games

Current rows from summary_metrics.csv · Oct '25 – Apr '26 · updated today

Model	Accuracy	Log Loss	Brier Score
Ensemble	66.31%	0.6218	0.2145
Logistic RegressionBEST	66.80%	0.6054	0.2096
LightGBM	64.33%	0.6300	0.2192

Key findings

Matched-sample benchmark

On the odds-matched sample, 10.6% advantage to the market with Brier scores of 0.161 for Vegas and 0.205 for the model. This subset should not be compared directly to the headline dashboard accuracy.

Broad agreement, not wild divergence

Model and Vegas agree on the predicted winner 77.3% of the time, meaning the model is broadly aligned with the market.

Edge thresholds do not produce profit

All edge thresholds produce negative ROI. Even at 10% edge, the ROI is -55.6%. This reflects the market's built-in vig.

Coverage matters

Coverage is 34% — odds data is only available from Feb '26 – Apr '26.

Methodology

Walk-forward integrity

Every prediction was made before tip-off using only data from prior games. No retraining on test games.

Closing-line benchmark

Predictions are compared against Pinnacle closing lines — the sharpest publicly available market. Closing lines represent the final consensus before tip-off.

Edge detection

An 'edge' game is one where the model's probability exceeds the Vegas implied probability by a configurable threshold. The ROI column shows simulated flat-bet returns at those odds.

Report generated May 20, 6:38 AM EDT. Re-generate with python -m src.pipeline.generate_market_report