The NFL Power Rankings Poll of Polls

Every Tuesday of the NFL season, a dozen outlets publish a power ranking. They disagree, usually in ways that feel arbitrary. So I did what you do with any panel of noisy opinions: I built a poll of polls, wrote it in Python, and made the disagreement the point.

Nine ballots, 32 teams, Week 2 of the 2026 season. Seattle and Buffalo are separated by 0.002 rank standard deviations — a statistical tie at the top. Pittsburgh is ranked as high as 13th and as low as 27th. That is not noise to be averaged away; it is the story.

Here is the consensus, with the spread across outlets shown for every team.

This week’s matchups

Every team plays in Week 2, so the slate is a clean test of what the panel thinks. All 16 games are priced below from the consensus alone, followed by a sandbox for building any matchup you like. The derivation, the calibration and its limits sit in the methodology further down.

A tie at the top is the correct answer

Buffalo and Seattle split the first-place votes four apiece, with Baltimore taking the ninth. The bootstrap gives Buffalo a 59% chance of topping a randomly re-drawn panel, but Seattle wins the point estimate, because Seattle’s worst ballot is fourth while Buffalo slips to fourth three times. The gap between the two scores is 0.002 z-units. For scale, the gap between second and third is 0.156 — nearly eighty times larger.

So the honest headline is not “Seattle is No. 1.” It is “Seattle and Buffalo are tied, and you should be suspicious of anyone who tells you otherwise after one week.”

The middle is where the argument is

The top of the table is boring for a statistical reason: between-team variance is enormous and the panel agrees closely, so the shrinkage reliability λ is 0.991 and the intervals are tight. The interesting teams are the ones nobody can place.

Chicago is the sharpest example. The Bears are fifth overall and have an 11-rank spread: two outlets have them second, one has them thirteenth. That is not a rounding error, it is a genuine disagreement about whether a 59-point opening week was signal or noise. The poll of polls reports the disagreement rather than resolving it by fiat.

The panel

Mean pairwise agreement across the panel is ρ = 0.926, with a range of 0.86 to 0.97. That is high enough for the consensus to mean something, and low enough that the consensus is not just one opinion copied nine times.

ESPN is deliberately absent. Every URL on the site — including the homepage — returns an empty HTTP 202 to a plain request, and its JSON API returns 403. Rather than scrape awkwardly or hand-transcribe a ballot, the panel documents the omission. When a headless renderer is worth the complexity, ESPN belongs in the panel.

How the matchups are priced

A ranking is an ordering, not a score, so pricing a game means giving the consensus a scale in points. That model has three constants and no free parameters:

margin(away @ home) = k * (consensus_home - consensus_away) + H
margin              ~ Normal(mean = margin, sd = sigma)
P(home wins)        = Phi(margin / sigma)

k converts a gap in consensus score into a gap in points, H is home-field advantage, and sigma is how far an actual margin typically lands from its expectation. Under the fitted values — k = 4.59 points per score unit, H = 2.56 points, sigma = 13.16 points — a team one full score-unit better than its opponent is about 4.6 points better on a neutral field, and the 13.2-point spread of outcomes is what keeps a 7-point edge from being a 90% win probability.

Those constants are not chosen by eye. tools/calibrate.py fits them from 6,719 completed regular-season games across 2000–2025:

1. Build each team’s strength from that season’s point differentials, standardised — and leave the game being predicted out of the ratings that predict it.
2. Fit k and H by least squares of the realised margin on the rating difference. Points are measured in points, so this fixes the scale directly.
3. Take sigma as the residual standard deviation, which is precisely the spread the probability step needs.

One methodological trap worth naming

The obvious approach is to grid-search all three constants by minimising log-loss on completed games. It produces beautiful probabilities and a nonsense scale, because the win/loss likelihood only identifies the ratios k/sigma and H/sigma. The optimum sits on a flat ridge, and the grid search returns whichever ridge point it happens to touch: a narrow grid gave k = 5.7, sigma = 15.5, and widening the same grid gave k = 8.25, sigma = 21.1 — identical log-loss, but the second would project 21-point margins between ordinary teams. The scale has to come from the quantity that carries units. That is why the fit is staged rather than joint, and why log-loss appears below as a check instead of an objective.

A second trap, caught the same way: without the leave-one-out step, a game helps set its own ratings and the slope inflates — k = 5.66 where the honest number is 4.78 for 2025 alone.

Does it work?

Graded on those 26 seasons, the model lands where a coin flip and the closing line bracket it:

It picks 66.8% of games correctly, and it is well calibrated: in the 2025 season, games it called at 80–90% went 67% of the time in the 80–90% band, and 90%+ calls hit 92%.

The honest comparison is the closing line, and the model is close to it — which is the expected result, not a triumph. Nine thoughtful rankings contain most of what a market knows, plus noise, and no amount of statistical care extracts information that was never in the ballots.

What this is not

This is not a betting model, and it should not be read as one. It uses only what the nine rankings say. No injury reports, rest, travel, weather, or market information enters a published number; closing lines appear in the pipeline solely as the independent benchmark in the table above. It also inherits every blind spot of the panel: if all nine outlets misjudge a team the same way, the probability is confidently wrong. sigma is sized to absorb that, which is why a decent edge still leaves a live underdog.

How the numbers are made

The four steps, in order. Full derivations with citations are in the pipeline’s METHODOLOGY.md.

1. Standardise. A ballot is a permutation of 1–32, so its mean (16.5) and standard deviation (9.30) are known analytically. Every rank becomes a z-score with the sign flipped so positive is better. A source that spreads teams out and one that clusters them end up on the same scale, so neither gains influence from style.

2. Pool. Sources are weighted by the credibility priors in sources.json (all between 0.75 and 1.00), which keeps any one outlet from swinging the result.

3. Measure the uncertainty two ways. A jackknife drops one outlet at a time and recomputes the consensus; how much a team’s score moves is that team’s sampling variance. Separately, a split-half procedure correlates the consensus computed from two random halves of the panel, then steps the correlation up with the Spearman-Brown formula. That answers the question every poll of polls should answer and almost none do: how many independent voices is this panel actually worth? For these nine outlets, ρ between halves is 0.983 and the effective count is 9.0 — the panel is unusually independent. When a panel is not (syndicated copies, or two desks from one outlet), the effective count drops and the published intervals widen automatically.

4. Shrink, then bootstrap the ranks. Empirical-Bayes shrinkage pulls each team’s pooled score toward the panel mean by a reliability factor estimated from the data itself. It keeps a team that exactly one outlet loves from jumping the consensus. Rank intervals and P(No. 1) come from 4,000 bootstrap resamples over the sources, so a position that depends on which outlets are in the room shows up as a wide interval.

One honest limitation: this measures the panel, not the truth. If all nine outlets share the same blind spot, the consensus shares it too, and no amount of clever weighting fixes that. The agreement matrix is the early-warning system — if pairwise correlations ever drift toward 0.99, it means the panel has stopped being nine opinions.

Reproduce it every week

The whole thing is one pure-standard-library Python package — no numpy, no pandas, no requests. The pipeline lives in analysis/nfl-power-rankings/: nine parsers, a validation gate, the estimator, and the report generator.

cd analysis/nfl-power-rankings
python3 -m unittest discover -s tests   # 50 tests, offline
python3 run.py --validate               # fetch, parse, validate — write nothing
python3 run.py                          # full run, refreshes this page's data

Two documents matter if you want to run it yourself each week:

• SOURCES.md is the replication manual. For all nine outlets it records the exact URL pattern, publication cadence, HTML structure, the parser that reads it, and the specific traps — USA Today’s slug week lags the edition number, FOX writes its list backwards from 32 to 1, NFL.com’s movement widget prints a number right after the rank, Sharp’s URL never changes so freshness has to be checked by hand. It also has a six-step weekly checklist and a worked example of adding a tenth source.
• METHODOLOGY.md is the statistics. Jackknife variance, Spearman-Brown reliability, method-of-moments variance decomposition, the shrinkage factor, the bootstrap, and a section on what the method does not claim.

Every run caches the raw HTML for each source and records its SHA-256 in consensus.json, so any published week can be re-derived byte-for-byte. The validation gate refuses to publish unless every ballot contains all 32 clubs exactly once, no two ballots are identical syndicated copies, and at least five complete sources are available. A parser that grabs a sidebar fails the build instead of quietly contributing a wrong number.

Keeping this honest as the season runs

The interesting part is not the Week 2 numbers, which will be wrong somewhere by Sunday night. It is whether the calibration holds. Each run writes out/matchups.csv with one row per game — projected margin, win probability, pick, and the closing line — and the columns for the final score are left empty on purpose. When the games are played, the scores get filled in and the same metrics that graded the historical fit (log-loss, Brier score, accuracy, and the calibration curve) can be recomputed on this season’s results.

That also gives the scale a weekly chance to fail. If k drifts, or the model’s log-loss pulls away from the closing line’s, that is evidence that the panel’s consensus has stopped carrying information the market doesn’t already have — which is worth reporting, and worth knowing before it becomes a comfortable assumption.

I will keep this updated through the season — the same nine outlets, the same estimator, with the week’s URL list rotated in sources.json. The interesting output will not be the rankings themselves but the reliability statistics week to week: whether the panel tightens as the season reveals teams, which clubs the consensus is genuinely unsure about, and whether the head-to-head model stays within touching distance of the market.