The problem with "most likely" outcomes (an NFL playoffs discussion)

On the final Sunday of the NFL regular season (yesterday, for those who weren't paying attention), there's always a number of moving parts as we try to figure out who is going to make the playoffs and who isn't (and also, who is going to be seeded where). Figuring it out is often a challenge, which is why it's nice to have a handy guide at your disposal to help you through the morass.

Luckily, our friends over at Deadspin were nice enough to provide just that, which was an immeasurable help to me as I sat around and rooted for the Patriots and made myself fatter (thanks to Sam Adams and some homemade lasagna). A Patriots win and a Texans loss meant that my Patriots earned themselves the #2 seed and a first-round bye, which was interestingly contrary to what Deadspin had told me to expect. To wit:

The most likely scenario [in the AFC] is that every team which has something at stake wins—they're almost invariably playing teams that don't—and thus the playoff order is exactly what you see above [Texans, Broncos, Patriots].

That line, when I first read it at 2pm or so, stuck with me as I watched the afternoon's games. What does "most likely scenario" really mean? It turns out that the way that we define our terms has an important impact on the way that we understand and respond to the world before us. That's what I'm about to explain.

Yes, it's true, in each individual game, it's "most likely" that the favorite will win—that's what being a favorite means. Therefore, when you start stringing together potential scenarios, the "most likely combination of outcomes" is, indeed, the combination in which all of the favorites win their games. But that doesn't necessarily make it the "most likely scenario"—there's a subtle but very important difference. Bear with me for a second here, because I'm about to get nerdy.

Let's start from the top here, considering a three-game sample. Let's say that each of the three top seeds coming into yesterday (Texans, Broncos, Patriots) had a 60% chance of winning their game (in the grand scheme of things in the NFL, that's a pretty high probability). To determine the likelihood of ALL THREE of them winning their games, which Deadspin said was the "most likely scenario", we just need to multiply the probabilities. In this case, 60% x 60% x 60% = 21.6% , so the likelihood of all the favorites winning was a little less than 1 in 4 odds.

There are 8 possible groupings of winners in this scenario (Texans/Broncos/Patriots would be one, Colts/Broncos/Patriots would be another, Texans/Chiefs/Dolphins a third, etc, etc, etc), and of those 8 possible groupings, the one where the favorites all win is indeed, as we said, the "most likely combination of outcomes". Here's a super-nerdy chart that shows that point, using the 60% probabilities that I used above.

When you look at it this way, you start to see that the "most likely scenario" isn't that all three teams will win, but that one of the other seven scenarios will occur (in fact, the "at least one upset" scenario is more than three times as likely here, with probability 78.4%).

Sure, any one of those individual outcomes is less likely than the individual outcome of "no upsets", but the reality of the matter is quite different. When we start to group the possible outcomes, we see things with a little bit more clarity. I think that a more realistic way of presenting the available data is the following:

Probability of exactly one upset:                       43.2% 

Probability of exactly two upsets:                     28.8%  

Probability of zero upsets:                            21.6% 

Probability of three upsets:                                6.4%

When you group the scenarios this way, you can see that all three teams doing what they're "supposed" to do is, in fact, far from the "most likely scenario" (NOTE: in order for it to become the "most likely scenario" the way I define it, the probability of each favorite winning would have to be more than 75%, as opposed to the 60% that I am using; I find 75% to be way too high for any NFL game). The statistics say that we should probably expect at least one upset, and that "exactly one upset" is the most likely scenario—which, unsurprisingly, is exactly what we ended up with.

Of course, in advance, we can't possibly know which game was likely to produce the upset, but it's almost beside the point. What we can know is that the more times we flip a coin, no matter how lopsided toward "heads" the coin may be, the more likely it becomes that it will eventually come up "tails".

Ultimately, the more independent variables (games) you start linking together, the more likely it is that your "most likely outcome" involves an upset (or a couple of upsets) somewhere along the way. In a sense, this is a similar statistical problem to the birthday problem, which I discussed here once before.

So, why does all of this matter? I'll make this part quick. Let's say you're the Broncos. You're sitting at home this week as the #1 seed, on your bye, trying to decide which team to prepare for (remember, the NFL re-seeds after the first round) while the Wild Card Weekend games are being played. With two games being played—Texans hosting Bengals, Ravens hosting Colts—there are four possible scenarios: (1) Texans and Ravens win, (2) Texans and Colts win, (3) Bengals and Ravens win, (4) Bengals and Colts win.

Assuming once again that the favorite has a 60% chance of winning, we get the following probabilities:

Texans/Ravens (Broncos play Ravens):                 36% 

Texans/Colts (Broncos play Colts):                       24% 

Bengals/Ravens (Broncos play Bengals):               24% 

Bengals/Colts (Broncos play Bengals):                  16%

So, using the same logic we used before, the most likely individual scenario is that both favorites (the Texans and Ravens) win their games, and so the Broncos should be preparing to play the Ravens next week... right?

But no, look again. Even though the "Bengals/Ravens" and "Bengals/Colts" scenarios are individually less likely than the "no upsets" scenario, they combine to be more likely. Because we've given the Bengals a 40% chance of winning their game, and because the Broncos will play the Bengals no matter what if they do indeed win, the Bengals are in fact the Broncos' most likely opponent. Sure, it's only by a small amount, but it's still relevant from a preparation standpoint—Denver should spend at least as much time watching Bengals film as Ravens film, if not more.

Any time we use statistics, we need to be careful with what we're really saying when we communicate our findings or beliefs. In the case of the Deadspin piece, the analysis in question wasn't wrong, it was simply imprecise (and possibly incomplete). When we as readers read that something is the "most likely" scenario, we're almost certainly hoping for something better than a 21.6% probability. Personally, I greatly prefer the much higher 43.2% probability that I ascribed to the "exactly one upset" scenario—I especially prefer it as a Patriots fan, whose team benefited greatly from the way things turned out on the field yesterday.

Good statistics (and good math, and good science, and good writing) requires that we be precise with our methods and our communication of our methods. If we're imprecise, we end up saying things that we don't really mean or that just aren't true.

[Deadspin]