Loading race intelligence…
Loading race intelligence…
Compare the model with simple baselines, inspect calibration, and follow the results of predictions saved before each race.
Prospective saved forecasts only. No predictions are invented for races before recording began. Metrics are separated by stage and model version. Latest pre-race revision per stage; race-balanced binary scores with race-cluster intervals. Sparse samples do not establish calibration.
Dots show race-balanced observed frequencies; the dashed line is perfect agreement. Sparse bins are uncertain. ECE: 6.7 percentage points.
| Probability bin | Predictions | Expected | Observed |
|---|---|---|---|
| 0–10% | 19 | 1.3% | 0.0% |
| 10–20% | 2 | 13.0% | 50.0% |
| 40–50% | 1 | 49.4% | 0.0% |
| Race | Race Brier ↓ | Cumulative Brier ↓ |
|---|---|---|
| United States Grand Prix | 0.046 | 0.046 |
0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast6 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Forecasts scored14 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Results · no recorded forecast0 saved snapshots
Open report card →Comparisons use the same races, stage, driver field and model version. Both forecasts and simple baselines must have been saved before the race. Older forecasts are preserved without retrospectively adding baselines.
Matched comparisons will appear after newly recorded forecasts reach completed races. More prospective weekends are needed to judge reliability.
This compares weather-aware models with the existing models and a no-weather ablation on identical race cohorts. Activation uses 2024 validation only. The previously inspected 2025 benchmark is for reporting; it is not an untouched final test.
Weather candidate enabledTraining 2019, 2020, 2021, 2022, 2023 · 75 races; validation 15; benchmark 16.
| Model / ablation | 2024 log loss ↓ | 2025 log loss ↓ | 2025 Brier ↓ |
|---|---|---|---|
| Existing model | 0.1429 | 0.1437 | 0.0428 |
| No weather · logistic | 0.1584 | 0.1771 | 0.0415 |
| Weather + conditions · logistic | 0.1577 | 0.1818 | 0.0414 |
| Weather + conditions · CatBoost · selected candidate | 0.1408 | 0.1385 | 0.0423 |
| No weather · CatBoost | 0.1459 | 0.1411 | 0.0426 |
Mean of three validation binary log losses improves by >0.001 over current serving and both matched no-weather ablations (logistic and CatBoost); no target worsens by more than 5% versus current serving.
Experimental fixed 72h-lead JMA weather plus prior nonfinish/circuit conditions. Historical weather publication is estimated with 6h release allowance. 2025 is a previously inspected development benchmark. No new source-shift certification. Unsupported stages or missing weather retain existing predictions.
Features: forecast temperature, precipitation amount, humidity, wind and driver-form interactions; prior nonfinish rates and circuit history. Forecast data is JMA GSM via Open-Meteo Previous Runs. Historical publication times are estimated with a six-hour release allowance. Missing forecasts and unsupported sprint formats do not receive invented values.
Raw scores are constrained to coherent winner / podium / top-10 marginals. A passed validation gate does not guarantee future calibration or improvement, including under the OpenF1 timing fallback.