Against Playoffs

The college football postseason winds down Thursday night when the Oklahoma Sooners and the Florida Gators finally meet in Miami. For a lot of people, the BCS national champion is going to be crowned illegitimately, because the system used to select the champion does not involve a playoff. The debate over a playoff in college football has raged for a long time, but as far as I'm aware, advocates of the system presently in place have always played defense in these debates. They say that the present system preserves the importance of the regular season; that it preserves the unique pageantry of the bowl season; they say that college football already has a playoff, and it's called the regular season.

What they don't say is that a playoff is a less efficient and less accurate method of determining the national champion than the present system is. Maybe they should.

The idea of crowning a champion is that we want to recognize the "best" team. Underlying this definitional matter is the assumption that each year, there is one, unique, "best" team. The hard part is identifying the best team. All systems will identify the wrong team as the best occasionally; the goal should be to maximize the percentage of seasons when the correct team is identified.

A playoff has its good points in this analysis. By giving more than two teams a chance to play for the title, a playoff increases the probability that the best team will make it into the mix. But there's a countervailing concern. The more teams allowed into the playoff, the more rounds it will require; the more rounds it requires, the more likely it is that the best team will lose.

That's right, because sometimes deciding it on the field means the better team loses. According to Coleman's rankings, which minimize the number of losses by a higher ranked team to a lower ranked team, the better team lost to the worse team no less than 9.22% of the time during the 2008 regular season. It's likely though that these so-called ranking violations occur more often than strictly mathematically necessary. Massey's consensus rankings—which are more in line with popular opinion than Coleman's are—place the percentage at 18.4%.

Note also that those numbers are across all games; where the opponents are more evenly matched, as they would be in a playoff scenario, the numbers would be significantly higher. So let's say that among the top 32 teams, the weaker beats the stronger with a probability of 32%. In each round, as the competition gets stiffer, the probability that the weaker beats the stronger increases by 4%, so that in the final round, the weaker team has a 48% chance of beating the stronger team. Obviously these assumptions are simplified (especially since they ignore the effects of seeding), but they seem reasonable enough.

So now let's estimate the probability that the best team will be included among those invited to the playoffs. Here, I'll fly a little more by the seat of my pants and say that the probability that the best team is among the top 2 is 50%; the probability that it's among the top 4 is 75%; the probability that it's among the top 8 is 87.5%; and so on.

So to calculate the probability that the best team is crowned champion, we multiply the probability that it has been invited by the probability that it wins all of its games once it's there. Let's see:

2 team playoff (current system):
P = 0.5 x 0.52 = 26%

4 team playoff:
P = 0.75 * 0.56 * 0.52 = 21.84%

8 team playoff:
P = 0.875 * 0.6 * 0.56 * 0.52 = 15.29%

16 team playoff:
P = 0.9375 * 0.64 * 0.6 * 0.56 * 0.52 = 10.48%

32 team playoff
P = 0.96875 * 0.68 * 0.64 * 0.6 * 0.56 * 0.52 = 7.37%

Based on my numbers, which clearly are open to debate and criticism, the current system gets it right more often than any of the extended playoff scenarios, crowning the correct champion a whopping 26% of the time. (One of the interesting things revealed by this exercise is just how pathetic even the best system is in the face of a problem this intractable).

A rejoinder from the pro-playoff camp at this point might be that the point of a playoff is not to crown the "best" team, but rather to crown the team that has accomplished something great, the team that is the last one standing atop a pile of defeated contenders. A playoff is a well-honed instrument for ensuring that this is the result: at the conclusion, one entrant is undefeated, and the rest have one loss. No arguments.

But that structure is wholly artificial. We all know that it's a mathematical impossibility to answer the question "Who's the best?" by answering the question "Who beat whom?" The regular season is full of contradictions and vicious circles that have to be wrestled into submission, not casually cast aside. A playoff is a machine that is not engineered to crown the best team; it is a machine engineered to put an end to arguments, without regard for whether the champion it produces is the best team or not.

What I will say for it, though, is that it would make for quite a spectacle.