The college football season is approaching its regular season climax, and as usual many teams -- including my beloved Iowa Hawkeyes -- are jockeying for a berth in a lucrative BCS bowl game. The system for determining this combines human (i.e., subjective) rankings and computer (i.e., objective) rankings. There are 6 different computer ranking systems in the BCS mix, and as usual they are the source of a lot of confusion and controversy. If you read NCAA message boards and blogs you often encounter heated discussions about how these mysterious computer ratings are calculated, and whether they include some sort of hidden "special sauce" or bias. Some of this focuses on the currently undefeated Hawkeyes, who are strangely ranked near the top of the computer polls despite having many close scrapes and come-from-behind wins. By comparison, Iowa is currently #7 or #8 in the subjective human polls.
The principle PR problem with the various BCS computer ratings (Sagarin, Massey, et al) is that the average fan has no idea where they spring from, let alone what's happening inside the various black boxes. The truth is that they really aren't that complicated. The rest of this post is my attempt to pull back the curtain on the how most BCS computer ratings are estimated for the casual fan. Don't fret, only a very modest amount of math is necessary. The rest of this post illustrates -- step by step -- how you can faithfully recreate the current BCS ratings via a downloadable Excel spreadsheet. I'll start with a simple intuitive example, then apply it to NCAA Division I-A games through October 31. Consider this your own free personal BCS computer.
Got the curiosity and a half hour? Let's go.
Stuff you will need
You can download the spreadsheet here (388.5K Excel file). In order to reproduce the examples that follow, you will need a version of Microsoft Excel with the Solver add-in. If you don't have Excel, you can replicate it with Sun's free OpenOffice Calc spreadsheet, but you will also need to download and install its non-linear Solver extension.
Ready? Start up Excel, and open the downloaded spreadsheet.
Simple Example
The spreadsheet tab labeled "Example" provides a simplified illustration of the mathematics underpinning most computer ranking systems. Here there are 5 teams: Arkasippi A&M, Bovine State, Cowlick College, Drunk U, and East Pennsyltucky. We don't know anything about how good these teams are, so let's start out naively assuming they all have a power rating of 100 (indicated in cells B7:B11). Click on the picture for a full screen view.
Underneath we see some actual game results, shaded in blue: Arkasippi A&M beats Bovine State 31-10 at home, Cowlick loses at home to Drunk U 14-17, and so on. That's reality. Now we have to compare reality to what the power ratings predict, which is shaded in yellow. For example in the Arkasippi A&M - Bovine State game (row 15), Arkasippi A&M won by an actual spread of 21. Our initial power ratings (Arkasippi A&M=100, Bovine St. = 100) predict a spread of 0. In other words our actual-predicted error was +21. The actual-minus-predicted error in Cowlick - Drunk U game (row 16) was -3, and so on. If we square the errors for each game, it ensures they are all positive. Now we can add up the squared errors for each game to get a "total sum of squares error" which is in cell J22.
Under our initial naive power ratings (everybody=100) we get an error total of 1015. Our goal at this point is to get that error number as small as possible, by adjusting the power ratings. From the Arkasippi A&M - Bovine State game it sure looks like Arkasippi is better, so let's drop the rating for Bovine State down from 100 to 95 and see what happens. Go to cell B8 and type in 95 (the prediction values in the yellow cells are linked to the power ratings and will update automatically).
Hey! Whaddayaknow. When we lower the power rating for Bovine State to 95, the error total drops from 1015 to 840. Ideally we would like to make total error 0, but that's a mathematical impossibility -- note that Arkasippi A&M beat Bovine State, Bovine State beat East Pennsyltucky, but... East Pennsyltucky beat Arkasippi A&M. A > B, B > E, E > A. That's called intransitivity, and no simple set of power rating can simultaneously predict all 3 outcomes.
Now, we could screw around all day typing different power ratings in cells B7:B11 to see which values the give us lowest total error, but fortunately Excel's Solver add-in will do that for us. To use it, select the Solver from Excel's Tools menu...
This will bring up the Solver dialog box.
Set the target cell to J22 (total square error), and click the button that tells Solver you want to minimize that cell. Enter the range B8:B11 as the cells that you want Solver to change. In this example, we are fixing the power rating of Arkasippi A&M at 100; all of the other power ratings will be adjusted relative to that number. Now click the "Solve" button.
Tada! Solver finds an error-minimizing set of power ratings: Cowlick College = 103.1, Drunk U = 101.5, Arkasippi A&M = 100, East Pennsyltucky = 90.8, Bovine State = 89.6. Total squared error now is 636.1, and no other set of team power ratings in cells B7:B11 can make that error any lower1.
But... what about the fact that some winning teams had the advantage of playing at home? It's fairly easy to include a home field advantage in the calculation, which you can see on the worksheet tab labeled "Add in Home Field."
In the list of power ratings there is now a value for home field at cell B12. As a starting guess, let's set it at 10. In the yellow prediction cells, the home field value is added to the power rating for each home team. Now let's call up Solver again and minimize square error by adjusting the team power ratings and home field value (B8:B12), again fixing Arkasippi A&M at 100.
And again, Ta-da. Factoring in Home Field advantage, we now get Arkasippi A&M = 100, Drunk U = 93.4, East Pennsyltucky = 89.8, Bovine State = 87.6, and Cowlick = 86.9. The Home Field advantage is estimated at 11.1. Factoring in home field advantage changes the power ratings of the teams, and it also lowers the total square error from 636 to 230.
The power ratings we've computed so far attempt to minimize error around actual point spreads, which creates a potential sportsmanship problem: it rewards teams for piling up "style points." For example, suppose Arkasippi A&M got a garbage touchdown in the last 30 seconds in its 31-10 win against Bovine State, but Bovine State took a knee on the 1 yard line in its 42-28 win against East Pennsyltucky. To take away the incentive for running up the score, we can treat every win as if it was a 1-point win. This is depicted on the worksheet tab labeled "Ignore Point Spread."
Here, notice the victory margin in each game is treated as 1 point; a win is a win is a win. If we pull up Solver and minimize error on this worksheet, we get a new set of power ratings: Arkasippi = 100, Cowlick and East Pennsyltucky tied at 99.67, Bovine State and Drunk U. tied at 99.33. Home field is estimated at 0.66.
"Okay, pal, but what about the real world?" Thought you'd never ask. If you click on the worksheet tab labeled "Real Data" you'll see this method applied to actual Division I-A college games. Instead of 5 hypothetical teams, there are power ratings for all 120 Division I-A teams, an unspecified Non-Division I-A team, plus Home Field. These are contained in rows 4-125. As a naive starting guess, power ratings are set at 100 with home field at 10.
Underneath, starting in row 134, you'll see the outcome of all 538 games involving Division I-A teams through October 31 (blue), predicted spread under the power ratings (yellow), and the error calculation (orange). Total Square Error is in cell M132.
Just as in the simple example, we can pull up Solver and minimize total point spread square error (cell M132) by adjusting the power ratings (cells B5:B125). Air Force is fixed at 100, and the other ratings will be adjusted relative to Air Force.
Depending on how fast your computer is, this may take a few minutes due to the number of ratings being adjusted. When it's finished, you'll see new power ratings for each team. If you sort them them in descending order of power ratings you should get the following Top 25:
| Rank | TEAM | Rating |
| 1 | Texas | 122.6245 |
| 2 | Florida | 121.7155 |
| 3 | Alabama | 117.0108 |
| 4 | Oregon | 115.8804 |
| 5 | Cincinnati | 115.6058 |
| 6 | Oklahoma | 115.0208 |
| 7 | Boise St | 112.8958 |
| 8 | TCU | 112.8537 |
| 9 | Virginia Tech | 111.4975 |
| 10 | Nebraska | 109.4934 |
| 11 | Southern Cal | 108.6608 |
| 12 | Tennessee | 107.8572 |
| 13 | Clemson | 107.8374 |
| 14 | Ohio State | 106.9607 |
| 15 | LSU | 106.7057 |
| 16 | Penn State | 106.5113 |
| 17 | Pittsburgh | 106.345 |
| 18 | Georgia Tech | 106.1382 |
| 19 | Texas Tech | 106.0698 |
| 20 | Arkansas | 105.0912 |
| 21 | Mississippi | 103.9505 |
| 22 | Boston College | 103.7563 |
| 23 | Miami FL | 103.7121 |
| 24 | California | 103.5901 |
| 25 | Auburn | 102.133 |
Home Field advantage comes in at 2.83. (Complete rankings are on the worksheet tab labeled "Real Ratings").
Okay, but whoah... 3-loss Oklahoma at #6? Tennessee #12? Undefeated Iowa not even in the top 25 (they're down at #27)? Keep in mind these ratings were all about minimizing error around actual point spreads, and style points are positively valued. Close losses can be compensated with blowouts wins (see e.g., Oklahoma).
What happens when we ignore margin of victory? That's the situation depicted in the worksheet tab labeled "Real Data, No Point Spread." The data here are identical to the previous sheet, except all wins are treated as 1-point wins. All that matters is victory. Let's pull up Solver again and minimize the total square error on this sheet (cell M132) by adjusting the Power ratings of the teams (B5:B125), again using Air Force = 100 as the comparison point.
Voila... the Top 25 ignoring the point spread is now
| Rank | TEAM | Rating |
| 1 | Florida | 101.15200 |
| 2 | Iowa | 101.00696 |
| 3 | Oregon | 101.00083 |
| 4 | Texas | 100.98775 |
| 5 | Cincinnati | 100.96969 |
| 6 | Alabama | 100.95487 |
| 7 | TCU | 100.89113 |
| 8 | Georgia Tech | 100.87334 |
| 9 | Southern Cal | 100.81186 |
| 10 | LSU | 100.80036 |
| 11 | Boise St | 100.76784 |
| 12 | Miami FL | 100.66319 |
| 13 | Pittsburgh | 100.65212 |
| 14 | Utah | 100.65119 |
| 15 | Houston | 100.61820 |
| 16 | Arizona | 100.61586 |
| 17 | Penn State | 100.58206 |
| 18 | Virginia Tech | 100.57730 |
| 19 | California | 100.54040 |
| 20 | Wisconsin | 100.53751 |
| 21 | Notre Dame | 100.53649 |
| 22 | Ohio State | 100.45569 |
| 23 | Oklahoma St | 100.44111 |
| 24 | Brigham Young | 100.43143 |
| 25 | South Florida | 100.41303 |
Hmmm... Iowa, #27 in the pure point spread rankings, is now #2; Close game losers Oklahoma, Tennessee and Clemson have suddenly disappeared from the top 25. This ranking closely resembles team rankings in the current week BCS computer polls, and for good reason: the official BCS computer polls are not allowed to look at margin of victory, and for the most part they all use a variant of this error minimization approach2.
Simplifying this Stuff
There's nothing really wrong about the methodology used in the BCS computer polls. They are completely objective in the sense that they focus only on accounting for the outcomes of games that have already been played. If you switched the name "Oregon" with "Alabama" everywhere in the spreadsheet, the Power Ratings would be switched for those 2 teams as well. The biggest PR issue with Computer Power Ranking is their opacity -- it's hard to explain them in 2 minutes or less.
But... there is a very simple way to approximate these rankings without having to resort to fancy stuff like iterative error minimization. It goes something like this:
1. Every team starts off the season with 0 points.
2. Teams earn points for every victory: 1 point for a win at home, 2 points for a neutral site win, 3 points for a road win.
3. Teams earn additional point for each Division I-A season victory by the teams it beats.
That's about as simple as it gets: any fan can easily double check his team's points by following the schedule of his team and the teams it defeats. It rewards strength of schedule for teams who win, against opponents who win, and ignores style points. It also approximates the results of the BCS computers.
Iowa, for example, notoriously squeaked out a home win against Northern Iowa in their opening game. So far this season Northern Iowa hasn't beaten a single Division I-A opponent, so that game was worth only 1 point to Iowa. The Hawkeyes followed that with a win at Iowa State. ISU has won 5 games this season but only 4 against Division I-A competition. Iowa so far has earned 7 total points for that win; 3 points for the road win plus 4 points for ISU's D-IA wins. Each time ISU wins another game, another point accrues to Iowa. In total Iowa has thus far earned 48 points - 4 road wins, 5 home wins, and the teams it has beaten have won a total of 31 victories against Div I-A competition.
How would the current rankings look under that simple scoring system?
| Rank | Team | Total Points |
| 1 | Iowa | 48 |
| 2 | Florida | 46 |
| 3 | Texas | 44 |
| 4 | Alabama | 42 |
| 5 | TCU | 40 |
| 6 | Cincinnati | 39 |
| 6 | Oregon | 39 |
| 8 | Georgia Tech | 38 |
| 9 | Ohio State | 36 |
| 9 | Penn State | 36 |
| 11 | Boise St | 35 |
| 11 | Southern Cal | 35 |
| 13 | Houston | 34 |
| 13 | LSU | 34 |
| 13 | Pittsburgh | 34 |
| 16 | Miami FL | 33 |
| 17 | Virginia Tech | 30 |
| 18 | Notre Dame | 29 |
| 19 | Temple | 28 |
| 20 | Brigham Young | 27 |
| 20 | Utah | 27 |
| 20 | Wisconsin | 27 |
| 20 | Auburn | 27 |
| 20 | Idaho | 27 |
| 25 | Troy | 26 |
Through the top 18 or so, it gets reasonably close to the current BCS computer rankings. Iowa has an advantage in the fact it has played 9 games so far, compared to 8 for Florida and most other Division I-A teams, and so has had one more opportunity to earn win points. If points are calculated on a per-games-played basis, the ranking looks like this:
| Rank | Team | Points per game |
| 1 | Florida | 5.75 |
| 2 | Texas | 5.50 |
| 3 | Iowa | 5.33 |
| 4 | Alabama | 5.25 |
| 5 | TCU | 5.00 |
| 6 | Cincinnati | 4.88 |
| 6 | Oregon | 4.88 |
| 8 | Boise St | 4.38 |
| 8 | Southern Cal | 4.38 |
| 10 | Houston | 4.25 |
| 10 | LSU | 4.25 |
| 10 | Pittsburgh | 4.25 |
| 13 | Georgia Tech | 4.22 |
| 14 | Miami FL | 4.13 |
| 15 | Ohio State | 4.00 |
| 15 | Penn State | 4.00 |
| 17 | Virginia Tech | 3.75 |
| 18 | Notre Dame | 3.63 |
| 19 | Arizona | 3.57 |
| 20 | Temple | 3.50 |
| 21 | Brigham Young | 3.38 |
| 21 | Utah | 3.38 |
| 21 | Wisconsin | 3.38 |
| 24 | Troy | 3.25 |
| 25 | California | 3.13 |
| 25 | Clemson | 3.13 |
| 25 | Oklahoma St | 3.13 |
A close echo of the BCS computers without all the big math complication. The surprise outlier is Temple, but the Owls are 6-2 with 3 road wins, against teams that have won collectively 16 games against D-IA competition.
Footnotes
1. The minimum square error power rating estimates are unique up to a scaling constant. For example if you subtracted 50 from every team's power rating, the rescaled estimates would exactly reproduce the predicted game spreads, and therefore result in the same error. That's why one team power rating is fixed and the rest are estimated relatively. For you math freaks, the square error function can be shown to be convex through the Hessian matrix, which is positive definite (proof omitted).
2. One variation includes giving a heavier weight to error on recent games; in other words making it more important to minimize error on last week's games versus the games 2 weeks ago, etc. The only BCS computer poll that reportedly doesn't have an error objective function is Billingsley, which is said to be a rule based point system -- rewards teams for road victories against higher ranked opponents, etc. -- that tries to represent the logic of a human poll voter. As a result the Billingsley rankings are often at odds with the other BCS polls and closer to the human polls.
