Before running down DYAR numbers from Super Bowl XLIV, Bill Barnwell gives his thoughts on Sean Payton's aggressive and intelligent play calls.
12 Nov 2008
by Brian Fremeau
The Fremeau Efficiency Index principles and methodology can be found here. Like DVOA, FEI rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. Unlike DVOA, it is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.
FEI is the opponent-adjusted value of Game Efficiency, a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. Like DVOA, it represents a team's efficiency value over average. Strength of Schedule is calculated from a privileged perspective (explained here) and represents the likelihood that an Elite team (top 5) would post an undefeated record against the given team's opponents to date.
The following ratings are calculated based on data from all FBS games played through Saturday, November 8. Only games between FBS teams are considered.
| Rank | Team | Record | FEI | Last Week | vs. Top 10 | vs. Top 40 | GE | GE Rank | SOS | SOS Rank |
| 1 | Florida | 8-1 | 0.310 | 1 | 0-0 | 2-1 | 0.462 | 1 | 0.447 | 61 |
| 2 | Penn State | 8-1 | 0.292 | 2 | 1-0 | 3-1 | 0.330 | 7 | 0.382 | 47 |
| 3 | North Carolina | 6-2 | 0.290 | 4 | 0-0 | 5-2 | 0.177 | 16 | 0.248 | 27 |
| 4 | Texas | 9-1 | 0.274 | 3 | 1-1 | 3-1 | 0.358 | 4 | 0.241 | 25 |
| 5 | Texas Tech | 8-0 | 0.270 | 7 | 1-0 | 2-0 | 0.379 | 3 | 0.437 | 59 |
| 6 | USC | 8-1 | 0.260 | 5 | 1-0 | 4-0 | 0.407 | 2 | 0.501 | 68 |
| 7 | Alabama | 10-0 | 0.224 | 6 | 0-0 | 3-0 | 0.295 | 9 | 0.511 | 71 |
| 8 | Oklahoma | 8-1 | 0.223 | 9 | 0-1 | 2-1 | 0.337 | 6 | 0.377 | 44 |
| 9 | Florida State | 5-2 | 0.196 | 10 | 0-0 | 3-2 | 0.083 | 27 | 0.327 | 39 |
| 10 | Ohio State | 7-2 | 0.190 | 13 | 0-2 | 2-2 | 0.133 | 18 | 0.188 | 10 |
| 11 | Georgia Tech | 5-3 | 0.187 | 8 | 1-1 | 3-3 | 0.069 | 30 | 0.174 | 5 |
| 12 | Virginia Tech | 5-3 | 0.182 | 11 | 1-1 | 2-3 | 0.038 | 43 | 0.183 | 7 |
| Rank | Team | Record | FEI | Last Week | vs. Top 10 | vs. Top 40 | GE | GE Rank | SOS | SOS Rank |
| 13 | Ball State | 8-0 | 0.176 | 20 | 0-0 | 0-0 | 0.346 | 5 | 0.796 | 115 |
| 14 | Iowa | 5-4 | 0.174 | 22 | 1-0 | 2-3 | 0.129 | 19 | 0.232 | 24 |
| 15 | Pittsburgh | 7-2 | 0.164 | 21 | 0-0 | 2-1 | 0.099 | 25 | 0.477 | 63 |
| 16 | Missouri | 7-2 | 0.161 | 15 | 0-1 | 1-2 | 0.221 | 14 | 0.378 | 46 |
| 17 | Mississippi | 4-4 | 0.159 | 14 | 1-1 | 1-3 | 0.037 | 44 | 0.182 | 6 |
| 18 | Wake Forest | 6-3 | 0.143 | 18 | 1-0 | 4-1 | 0.025 | 53 | 0.325 | 37 |
| 19 | Boston College | 5-3 | 0.142 | 28 | 0-1 | 1-3 | 0.068 | 31 | 0.224 | 19 |
| 20 | Miami | 5-3 | 0.141 | 17 | 0-3 | 2-3 | 0.034 | 46 | 0.108 | 3 |
| 21 | Connecticut | 5-3 | 0.128 | 25 | 0-1 | 2-3 | 0.046 | 39 | 0.295 | 32 |
| 22 | Michigan State | 9-2 | 0.127 | 24 | 0-1 | 2-2 | 0.104 | 23 | 0.368 | 43 |
| 23 | Georgia | 7-2 | 0.125 | 16 | 0-2 | 1-2 | 0.045 | 41 | 0.205 | 14 |
| 24 | Boise State | 8-0 | 0.124 | 26 | 0-0 | 0-0 | 0.310 | 8 | 0.801 | 116 |
| 25 | Utah | 9-0 | 0.124 | 37 | 0-0 | 1-0 | 0.222 | 13 | 0.709 | 106 |
Adjusted Offensive Efficiency and Adjusted Defensive Efficiency are the opponent-adjusted values of Offensive Efficiency and Defensive Efficiency, explained here. Like FEI, the multiple-order adjustments are weighted according to both the strength of the opponent and the relative significance of the result; efficiency against a team's best competition faced is given more relevance weight. AOE and ADE represent a team's value over/under average. Positive AOE and negative ADE are the most valuable.
| Rank | Team | Record | AOE | AOE Rank | ADE | ADE Rank | OE | OE Rank | DE | DE Rank |
| 1 | Florida | 8-1 | 0.428 | 6 | -0.461 | 4 | 0.579 | 9 | -0.639 | 2 |
| 2 | Penn State | 8-1 | 0.554 | 3 | -0.392 | 10 | 0.490 | 13 | -0.467 | 7 |
| 3 | North Carolina | 6-2 | 0.269 | 20 | -0.512 | 1 | -0.037 | 60 | -0.366 | 14 |
| 4 | Texas | 9-1 | 0.551 | 4 | -0.371 | 13 | 0.959 | 2 | -0.151 | 42 |
| 5 | Texas Tech | 8-0 | 0.568 | 1 | -0.388 | 11 | 1.134 | 1 | 0.000 | 64 |
| 6 | USC | 8-1 | 0.409 | 7 | -0.414 | 6 | 0.495 | 12 | -0.666 | 1 |
| 7 | Alabama | 10-0 | 0.279 | 18 | -0.427 | 5 | 0.164 | 32 | -0.586 | 3 |
| 8 | Oklahoma | 8-1 | 0.556 | 2 | -0.235 | 27 | 0.899 | 4 | -0.135 | 44 |
| 9 | Florida State | 5-2 | 0.277 | 19 | -0.186 | 37 | 0.005 | 51 | -0.145 | 43 |
| 10 | Ohio State | 7-2 | 0.169 | 30 | -0.399 | 8 | -0.011 | 55 | -0.404 | 9 |
| 11 | Georgia Tech | 5-3 | 0.300 | 15 | -0.269 | 22 | -0.031 | 59 | -0.284 | 21 |
| 12 | Virginia Tech | 5-3 | 0.009 | 63 | -0.365 | 14 | -0.201 | 82 | -0.157 | 38 |
| Rank | Team | Record | AOE | AOE Rank | ADE | ADE Rank | OE | OE Rank | DE | DE Rank |
| 13 | Ball State | 8-0 | 0.324 | 13 | -0.310 | 17 | 0.791 | 5 | -0.308 | 19 |
| 14 | Iowa | 5-4 | 0.092 | 48 | -0.490 | 2 | -0.028 | 57 | -0.404 | 9 |
| 15 | Pittsburgh | 7-2 | 0.334 | 12 | -0.179 | 39 | 0.267 | 23 | -0.037 | 58 |
| 16 | Missouri | 7-2 | 0.383 | 9 | -0.129 | 47 | 0.705 | 6 | 0.041 | 73 |
| 17 | Mississippi | 4-4 | 0.298 | 16 | -0.220 | 32 | -0.012 | 56 | -0.125 | 46 |
| 18 | Wake Forest | 6-3 | -0.200 | 87 | -0.475 | 3 | -0.339 | 104 | -0.245 | 24 |
| 19 | Boston College | 5-3 | 0.149 | 37 | -0.411 | 7 | -0.081 | 63 | -0.489 | 6 |
| 20 | Miami | 5-3 | 0.075 | 54 | -0.015 | 63 | -0.201 | 81 | -0.074 | 53 |
| 21 | Connecticut | 5-3 | 0.149 | 38 | -0.351 | 15 | -0.028 | 58 | -0.307 | 20 |
| 22 | Michigan State | 9-2 | 0.163 | 31 | -0.202 | 35 | -0.108 | 65 | -0.239 | 25 |
| 23 | Georgia | 7-2 | 0.403 | 8 | -0.095 | 52 | 0.167 | 31 | -0.083 | 51 |
| 24 | Boise State | 8-0 | 0.158 | 33 | -0.210 | 33 | 0.334 | 18 | -0.557 | 4 |
| 25 | Utah | 9-0 | 0.106 | 45 | -0.229 | 31 | 0.053 | 46 | -0.408 | 8 |
The Week 11 FEI Ratings for all 120 FBS teams can be found here. Expanded FEI Ratings data can be found here.
Penn State certainly isn't the first team to lose a game without dropping in the FEI rankings, and I'm confident they won't be the last. The Nittany Lions' dream season fizzled away on Saturday at the hands of four-loss Iowa. Like Mississippi, FEI respected the Hawkeyes competitive losses in its other games even before the upset bid. Most of Penn State's opponents were victorious over the weekend as well, and those re-calibrated results keep them afloat.
Texas Tech made another big statement, slaughtering Oklahoma State, but they still hang behind the Longhorns and will need to beat Oklahoma in order to move ahead in FEI. The back-to-back wins have cast a big spotlight on Lubbock, Graham Harrell, and Michael Crabtree, and their resume is nice, but still just a little thin. Those victories remind me of a one-two punch delivered by Texas not too long ago.
The ACC/FEI romance is getting even more serious. North Carolina hasn't run a Big 12 South gauntlet, but they do boast more games (seven) and wins (five) against FEI top 40 competition than anyone in the FEI top 10. The Tar Heels' non-top 40 opponent? No. 42 Notre Dame.
In the process of calculating AOE and ADE, recall that I discard all garbage possessions and then extract each drive's offensive points earned based on national efficiency expectations for both the drive start and end field position. Now let's take a closer look at the leftovers. I prefer not to combine the efforts of all traditional special teams units (kickoff, kick return, punt, punt return, field goal, and extra point) into one metric. Instead, I am presenting a special team scoring efficiency and a field position efficiency metric independently.
The STSE metric is designed to measure the success of each team's " points-after" units, both those that trot onto the field following touchdowns and those that line up after offensive drives that stall in opponent territory. The post-touchdown attempts are pretty straightforward -- an extra-point kick is worth one full point on the scoreboard, but since the national success rate for these kicks is so high, I reward the offense with 6.958 points for their score, leaving 0.042 points to be earned per extra point attempt (and 1.042 points to be possibly earned for each two-point attempt).
Non-touchdown offensive drives are credited for advancing into opponent territory according to both national field goal attempt and punt frequency rates combined. For an offensive drive that stalls at the opponent 34-yard line, for instance, I measure the " points-after" success based not simply on the success rate of 51-yard field goal attempts, but rather divided by the sum of all 51-yard attempts and punts from the opponent's 34-yard line.
The FPE metric divides the cumulative field position score expectations of each team's drives over those of its opponents. Punt, kickoff, interception, and fumble returns for scores are also added into the cumulative FPE score expectation total though they are not, of course, part of an offensive drive. A long return to an opponent's five-yard line is valued at 5.471 score expectation points. If the same return makes it to the end zone, the FPE score value is, like any offensive touchdown, 6.958 points. FPE is a combination of offensive success moving the football, defensive success in killing drives, and special teams kick, punt and return execution -- all of the parts of the game not already accounted for in OE, DE and STSE.
Now, admittedly, separating STSE and FPE in this way is not perfectly precise. The decision to punt from an opponent's 30-yard line may be a wise strategic play even when a team has an excellent field goal kicking unit. But it does help outline the decision in score expectation terms. A team with average offense, defense and special teams units can expect to net 1.318 points from an attempt at the opponent's 30-yard line. Giving up the ball at the 30-yard line grants the opponent a drive-score expectation of 1.885 points for their next drive. Punting to the opponent's ten-yard line grants the opponent a score expectation of 1.239 points. If the punt is a touchback, the opponent score expectation elevates to 1.535 points. Factoring the strength of the opponent offense and the team's own defense changes the calculus, but you get the idea.
The following tables list the top 10 teams in STSE, Opponent STSE (by least efficient opponent special teams scoring), and FPE.
| Top 10 STSE | Top 10 Opponent STSE | Top 10 FPE | |||||||||||
| Rank | Team | STSE | FEI Rank | Rank | Team | Opp. STSE | FEI Rank | Rank | Team | FPE | FEI Rank | ||
| 1 | Florida State | 0.146 | 9 | 1 | USC | -0.168 | 6 | 1 | Wake Forest | 0.304 | 18 | ||
| 2 | Syracuse | 0.081 | 91 | 2 | Boise State | -0.105 | 24 | 2 | North Carolina | 0.299 | 3 | ||
| 3 | Cincinnati | 0.073 | 26 | 3 | Kentucky | -0.090 | 65 | 3 | Florida | 0.296 | 1 | ||
| 4 | Miami (OH) | 0.072 | 97 | 4 | Florida | -0.071 | 1 | 4 | Louisiana Tech | 0.263 | 108 | ||
| 5 | Arkansas State | 0.063 | 84 | 5 | East Carolina | -0.064 | 32 | 5 | TCU | 0.244 | 30 | ||
| 6 | Miami | 0.062 | 20 | 6 | Auburn | -0.062 | 52 | 6 | BYU | 0.229 | 61 | ||
| 7 | Utah | 0.059 | 25 | 7 | Middle Tennessee | -0.056 | 98 | 7 | South Carolina | 0.221 | 29 | ||
| 8 | UTEP | 0.053 | 77 | 8 | Ohio | -0.054 | 78 | 8 | Oklahoma State | 0.197 | 27 | ||
| 9 | Idaho | 0.050 | 118 | 9 | Alabama | -0.052 | 7 | 9 | Ohio State | 0.170 | 10 | ||
| 10 | West Virginia | 0.050 | 28 | 10 | Temple | -0.052 | 59 | 10 | Kentucky | 0.167 | 65 | ||
Florida State leads the way in STSE, having made all but one of 18 non-garbage field goal attempts this season, including four from more than 50 yards. An average unit would expect to score 30.3 points on those 18 possessions; FSU's Graham Gano has contributed 51 points with his leg.
The USC defense is stifling enough, limiting opponents to seven non-garbage touchdowns on the season (four of which were recorded by Oregon State). It doesn't seem fair that they are also limiting opponent STSE to the worst rate in the country. Opponents have only kicked four field goals on ten non-garbage attempts this year. The only successful kicks made against USC thus far were attempted inside the Trojans' 20-yard line.
Florida's defense has almost been USC's equal, and their command of FPE is devastating opponents. In addition to four non-garbage return score touchdowns, the Gators have started 18 possessions in opponent territory this season, reaching the end zone on 15 of those drives. Wake Forest has had 26 possessions begin on the opponent's side of the 50-yard line but has only reached the end zone on eight of those drives.
The complete STSE and FPE ratings for each team can be found here. None of these metrics are opponent-adjusted, in part because I haven't done enough research to determine how to make such adjustments. I'm not sure the opponent adjustments on these should be as influential as the offensive and defensive adjustments made for AOE and ADE, but I am welcome to suggestions.
(Ed. note: Actually, NFL special teams ratings are not adjusted for opponent either... I've never been able to do it in a way where the results seem to improve accuracy. -- Aaron Schatz)
4 comments, Last at 15 Nov 2008, 11:28pm by David227
Comments
Re: FEI Week 11 Ratings
Do you ever try to take into account bad officiating? For instance, in the Cocktail (Keg) Party there was a no-call holding and a bad spot on Tebow's run that was not handled through review. Did Richt complain to the SEC/officiating or was there any response about that play?
A week later (makeup call?) Percy Harvin clearly did not fumble (the ball was in control until it hit the ground) and in fact the ball even cleared the goal line. The two Vandy touchdowns could have been questionable (control gained with arm lying out-of-bounds, no call on what looked to be offensive pass interference as WR shoved off to get room for the other TD).
I think the Gators are playing very well and may be playing as the best team in the country (along with USC and maybe Texas Tech or the other Big 12 South teams). However, it also seems like (other than the Ole Miss game) things are bouncing in their favor and if they stop (or bounce the other way) they could have a much tougher time of it.
Re: FEI Week 11 Ratings
Pete, as far as I know, very few stat sights keep up with NFL officiating (FO does somehow). I do not believe that they make individual stat adjustments based on horrible calls (Ed Hochuli, week 2, anyone?). To do this for 50+ FBS games on a weekly basis would be IMPOSSIBLE, esp. since most of them take place on Sat. At least in this part of the year the NFL has 13 games during its reg. Sun slots, with one on Mon., Sun. PM, and one on Thurs. Considering that there are almost 4x that many (13) games on Sat., you would prob. need 20+ volunteers, and of course most of these will claim "bad" officiating when their team gets shafted on a questionable call that could prob. go either way. And since both replay systems have the standard of "clearly-the-ref-got-the-call-wrong" evidence to overturn a call, anything that is unclear will stand as called. (BTW, I think that this is the best standard). I don't see how adjusting for bad officiating could be done without screwing up the objective stats. (Not to mention, both the NCAA & NFL hate to admit that the ref got it wrong, even when it's pretty clear.)
Re: FEI Week 11 Ratings
Ooh, controversy! Not only is Washington State not the worst team in 1-A in terms of raw or adjusted Defensive Efficiency, they're not even the worst team in the State of Washington at either category. Given the second-fewest points they've allowed in a Pac-10 game this year is 58, does this mean FEI's focus on competitive possessions is underestimating just how bad Wazzu is, or are comparatively spectacular results like giving up only 39 to Oklahoma State and 45 to Baylor the cause? Or is it the really dreadful offense (120th in raw OffEff) making the defense look better than I think it is?
Re: FEI Week 11 Ratings
If the metrics continually place teams like North Carolina and Va Tech as one of the tops teams in the country and the continue to lose, then the metrics are not a very good predictor of the comparitive strength of teams.
Post new comment