Before running down DYAR numbers from Super Bowl XLIV, Bill Barnwell gives his thoughts on Sean Payton's aggressive and intelligent play calls.
05 Dec 2008
by Bill Connelly
There are basically four levels at which to look at college football statistics: the year level, the game level, the possession level, and the play-by-play level.
The year level is most often just looked at in terms of wins and losses. Team A went from 7-5 in Year A to 5-7 in Year B and 3-9 in Year C; consequently, Coach A was fired. There isn't much, statistically, in this.
The game level is extremely common. Team A is averaging 37.3 points per game. Quarterback B is averaging 298.7 passing yards per game. You can rank teams and players with this, but much of the underlying story is still left untold.
Football Outsiders readers know that the possession level is Brian Fremeau's specialty. His Fremeau Efficiency Index derives ratings and prediction from (non-garbage time) possession data.
That leaves the play-by-play level. It is the reason Varsity Numbers exists. As the college football regular season comes to a close, now is the time to start looking at what play-by-play data can tell you about the season you have just witnessed.
To do that, we will first revisit a couple of concepts discussed earlier in the life of this column. The first is S&P, which you can find defined in the Football Outsiders glossary. It is the OPS of college football statistics, combining efficiency (success rates) and explosiveness (Points Per Play).
The second is the "+" concept, a primitive attempt at a college-level look at DVOA-type comparisons discussed here. The formulas behind the "+" concept have been strengthened since the writing of that column (instead of looking at game totals and comparing them to opponents' per-game averages, it actually looks at per-play totals, eliminating the general per-game problem of small sample sizes), but the idea remains the same: Figure out a way to factor a team's strength of schedule into statistics to truly find the best offense, defense, and possibly overall team. You read "+" ratings the same way you would read OPS+ or something similar in baseball: 100 is exactly average, under 100 is bad, and over 100 is good.
(If you want to look at this as a DVOA figure of sorts, just subtract 100 and view it as a percentage. So a "+" figure of 122 could also be viewed as +22%. A "+" figure of 90 would be -10%. The Overall "+" figure below is the same way, only the base is 200 instead of 100.)
One caveat with this post: Play-by-play entry without a parser like the one we use in the NFL (and like the one we're in the middle of building for NCAA data) is time-consuming, and only the 2008 games involving BCS teams have been entered to date. True "+" ratings cannot be calculated until all games have been entered. So why are we discussing this topic today? Because it is conference championship weekend, and there is a lot of interesting data to share regarding conference play.
So let us dive into the world of BCS conference data. The tables below show you conference breakdowns of the following statistics:
Rushing S&P+ (offense and defense): A comparison of a team's rushing output to the output expected based on the number of rushes against the team's opponents.
Passing S&P+ (offense and defense): A comparison of a team's passing output to the output expected based on the number of passes against the team's opponents.
Close-Game S&P+ (offense and defense): This is an overall S&P+ figure for all plays run while a given game was "close." What makes a game "close" has been tweaked from previous Varsity Numbers columns as well. Previously, "close" simply signified a scoring margin within less than 17 points. This has been altered in the following ways:
Overall, this is a relatively minor change, but it does signify that an early 21-point lead is much "closer" than a 21-point lead in the second half.
Being that a team's performance while a game is still more or less winnable is what is most important (and when a team gives up a ton of yards and points when they're up 49-7, it really is not that important), the Close-Game "+" numbers are used for the next measure as well.
Overall "+": This is the measure by which the teams in these tables are ranked. It is simply Offensive Close-Game S&P+ plus Defensive Close-Game S&P+. In this category, 200 signifies average, not 100.
Projected Record: In theory, if you know how many EqPts per game Team A averages rushing and passing, and if you know how far Team B usually holds opponents above or below their season averages, then you can come up with a figure that represents Team A's likely output against Team B, and vice versa. Throw in a home-field adjustment, and you can project likely results and therefore a team's likely record based on their (conference) season averages.
In other words, if every team played at its average level in every game (which will obviously never happen), you can project what their record likely would have been. Differences between actual and projected records could be explained by far too many variables to count -- special teams breakdowns, good or bad luck, good or bad coaching, or maybe the simple fact that some teams get much better or worse as the season progresses.
So with all of these explanations and caveats in mind, here are the major "+" figures, rankings, and projected conference records for the six BCS conferences. Each category below does not work out to a perfect 100.0 average because each team ran a different number of plays and therefore has a different denominator.
| Big 12 Conference | |||||||||
| Team | Off. Rushing S&P+ | Def. Rushing S&P+ | Off. Passing S&P+ | Def. Passing S&P+ | Off. Close S&P+ | Def. Close S&P+ | Overall "+" | Proj. Record | Actual Record |
| Oklahoma | 104.6 | 108.6 | 128.9 | 118.6 | 129.0 | 113.7 | 242.7 | 8-0 | 7-1 |
| Texas | 107.9 | 121.8 | 113.7 | 121.1 | 105.4 | 133.9 | 239.3 | 6-2 | 7-1 |
| Texas Tech | 112.4 | 93.9 | 119.5 | 106.6 | 120.0 | 98.1 | 218.1 | 7-1 | 7-1 |
| Missouri | 115.6 | 115.7 | 112.7 | 99.4 | 107.7 | 109.9 | 217.6 | 6-2 | 5-3 |
| Oklahoma State | 108.9 | 106.1 | 121.4 | 103.1 | 109.3 | 107.9 | 217.2 | 4-4 | 5-3 |
| Nebraska | 105.0 | 99.7 | 105.0 | 100.9 | 103.1 | 103.4 | 206.5 | 6-2 | 5-3 |
| Kansas | 103.9 | 105.4 | 97.5 | 102.1 | 101.1 | 104.9 | 205.9 | 3-5 | 4-4 |
| Baylor | 96.6 | 108.2 | 78.7 | 99.4 | 90.3 | 100.1 | 190.4 | 2-6 | 2-6 |
| Kansas State | 90.1 | 89.5 | 81.4 | 94.8 | 88.6 | 87.9 | 176.5 | 1-7 | 2-6 |
| Colorado | 78.1 | 104.5 | 67.2 | 102.6 | 70.5 | 102.4 | 172.9 | 2-6 | 2-6 |
| Texas A&M | 83.2 | 84.5 | 86.2 | 88.6 | 77.6 | 90.3 | 167.9 | 2-6 | 2-6 |
| Iowa State | 91.6 | 88.1 | 83.5 | 72.7 | 84.2 | 80.6 | 164.7 | 1-7 | 0-8 |
Thoughts and observations:
| Southeastern Conference (SEC) | |||||||||
| Team | Off. Rushing S&P+ | Def. Rushing S&P+ | Off. Passing S&P+ | Def. Passing S&P+ | Off. Close S&P+ | Def. Close S&P+ | Overall "+" | Proj. Record | Actual Record |
| Florida | 148.0 | 101.9 | 135.5 | 115.8 | 143.5 | 116.6 | 260.0 | 8-0 | 7-1 |
| Alabama | 115.5 | 130.3 | 116.8 | 117.6 | 115.1 | 118.6 | 233.7 | 7-1 | 8-0 |
| Mississippi | 95.1 | 119.4 | 125.8 | 108.5 | 110.9 | 104.6 | 215.5 | 5-3 | 5-3 |
| Tennessee | 103.9 | 136.9 | 80.3 | 110.9 | 93.4 | 121.0 | 214.5 | 3-5 | 3-5 |
| Georgia | 108.0 | 89.9 | 134.2 | 83.0 | 122.7 | 88.2 | 210.9 | 4-4 | 6-2 |
| LSU | 113.6 | 89.9 | 114.6 | 93.7 | 115.3 | 91.9 | 207.2 | 4-4 | 3-5 |
| South Carolina | 80.0 | 97.7 | 98.8 | 111.3 | 97.2 | 105.4 | 202.6 | 4-4 | 4-4 |
| Arkansas | 109.6 | 111.3 | 104.4 | 95.4 | 109.6 | 92.5 | 202.0 | 3-5 | 2-6 |
| Vanderbilt | 85.9 | 93.4 | 85.2 | 99.0 | 86.2 | 94.0 | 180.2 | 3-5 | 4-4 |
| Auburn | 71.6 | 103.2 | 76.4 | 94.5 | 77.8 | 100.5 | 178.4 | 2-6 | 2-6 |
| Kentucky | 92.2 | 90.7 | 84.1 | 92.4 | 85.3 | 92.4 | 177.7 | 2-6 | 2-6 |
| Mississippi State | 76.5 | 94.3 | 65.1 | 83.3 | 76.0 | 91.1 | 167.2 | 3-5 | 2-6 |
Thoughts and observations:
| Atlantic Coast Conference (ACC) | |||||||||
| Team | Off. Rushing S&P+ | Def. Rushing S&P+ | Off. Passing S&P+ | Def. Passing S&P+ | Off. Close S&P+ | Def. Close S&P+ | Overall "+" | Proj. Record | Actual Record |
| Florida State | 123.7 | 95.6 | 102.7 | 117.1 | 117.1 | 104.7 | 221.8 | 7-1 | 5-3 |
| Georgia Tech | 122.4 | 94.4 | 111.1 | 99.7 | 119.4 | 97.9 | 217.3 | 4-4 | 5-3 |
| Clemson | 88.5 | 108.8 | 102.2 | 122.5 | 96.7 | 115.3 | 212.1 | 4-4 | 4-4 |
| Virginia Tech | 96.0 | 117.7 | 84.3 | 110.0 | 91.4 | 116.0 | 207.4 | 4-4 | 5-3 |
| Boston College | 101.9 | 107.1 | 102.9 | 99.4 | 103.4 | 103.6 | 207.0 | 4-4 | 5-3 |
| North Carolina | 93.7 | 102.1 | 112.5 | 104.5 | 103.0 | 103.5 | 206.5 | 5-3 | 4-4 |
| Wake Forest | 79.2 | 113.3 | 88.3 | 104.2 | 85.4 | 112.9 | 198.3 | 4-4 | 4-4 |
| N.C. State | 113.2 | 79.6 | 112.4 | 83.5 | 114.6 | 81.2 | 195.8 | 4-4 | 4-4 |
| Maryland | 96.6 | 93.6 | 107.1 | 92.4 | 101.4 | 91.9 | 193.3 | 3-5 | 4-4 |
| Virginia | 93.5 | 103.2 | 95.4 | 100.6 | 90.5 | 100.4 | 190.9 | 4-4 | 3-5 |
| Miami | 110.4 | 84.5 | 106.7 | 89.4 | 104.0 | 84.7 | 188.7 | 4-4 | 4-4 |
| Duke | 84.7 | 105.3 | 98.9 | 78.3 | 92.8 | 89.1 | 181.9 | 1-7 | 1-7 |
Thoughts and observations:
| Big Ten Conference |
|||||||||
| Team | Off. Rushing S&P+ | Def. Rushing S&P+ | Off. Passing S&P+ | Def. Passing S&P+ | Off. Close S&P+ | Def. Close S&P+ | Overall "+" | Proj. Record | Actual Record |
| Penn State | 117.8 | 116.6 | 122.7 | 117.3 | 119.6 | 116.6 | 236.2 | 6-2 | 7-1 |
| Ohio State | 104.8 | 108.2 | 124.1 | 125.0 | 117.3 | 118.1 | 235.4 | 8-0 | 7-1 |
| Iowa | 116.0 | 128.2 | 111.1 | 110.2 | 115.5 | 114.6 | 230.1 | 7-1 | 5-3 |
| Illinois | 86.0 | 98.9 | 83.2 | 93.1 | 116.7 | 97.4 | 214.1 | 4-4 | 3-5 |
| Wisconsin | 113.7 | 94.8 | 89.5 | 109.1 | 101.6 | 101.9 | 203.5 | 4-4 | 3-5 |
| Northwestern | 89.3 | 95.3 | 107.4 | 96.5 | 98.3 | 97.7 | 195.9 | 5-3 | 5-3 |
| Michigan State | 85.1 | 100.2 | 109.7 | 101.7 | 95.8 | 96.6 | 192.5 | 4-4 | 6-2 |
| Purdue | 94.4 | 96.2 | 93.1 | 102.1 | 89.3 | 98.7 | 187.9 | 3-5 | 2-6 |
| Minnesota | 79.9 | 101.3 | 89.2 | 91.7 | 85.5 | 95.6 | 181.1 | 3-5 | 3-5 |
| Michigan | 93.2 | 99.7 | 71.5 | 92.9 | 82.9 | 95.0 | 178.0 | 0-8 | 2-6 |
| Indiana | 86.0 | 78.9 | 83.2 | 84.1 | 90.6 | 85.5 | 176.1 | 0-8 | 1-7 |
Thoughts and observations:
| Pacific 10 Conference |
|||||||||
| Team | Off. Rushing S&P+ | Def. Rushing S&P+ | Off. Passing S&P+ | Def. Passing S&P+ | Off. Close S&P+ | Def. Close S&P+ | Overall "+" | Proj. Record | Actual Record |
| USC | 118.3 | 121.6 | 134.4 | 147.8 | 125.0 | 133.4 | 258.4 | 8-0 | 7-1 |
| Oregon | 120.4 | 119.4 | 112.0 | 93.2 | 120.4 | 112.5 | 232.9 | 7-2 | 7-2 |
| California | 104.5 | 124.8 | 97.7 | 123.7 | 99.6 | 123.0 | 222.6 | 5-3 | 5-3 |
| Oregon State | 102.6 | 112.4 | 120.4 | 100.4 | 107.6 | 104.8 | 212.4 | 5-4 | 7-2 |
| Arizona | 112.1 | 97.0 | 108.9 | 102.4 | 111.2 | 99.4 | 210.6 | 4-4 | 4-4 |
| UCLA | 73.1 | 107.5 | 86.7 | 138.9 | 84.0 | 117.7 | 201.7 | 5-3 | 3-5 |
| Stanford | 110.9 | 93.2 | 111.7 | 87.0 | 111.7 | 86.7 | 198.4 | 3-6 | 4-5 |
| Arizona State | 77.0 | 125.2 | 98.8 | 97.3 | 84.9 | 108.3 | 193.1 | 4-4 | 4-4 |
| Washington | 74.0 | 82.7 | 83.9 | 81.1 | 72.7 | 77.8 | 150.5 | 1-7 | 0-8 |
| Wash. State | 66.9 | 79.1 | 58.7 | 75.8 | 62.9 | 72.3 | 135.2 | 0-9 | 1-8 |
Thoughts and observations:
| Big East Conference |
|||||||||
| Team | Off. Rushing S&P+ | Def. Rushing S&P+ | Off. Passing S&P+ | Def. Passing S&P+ | Off. Close S&P+ | Def. Close S&P+ | Overall "+" | Proj. Record | Actual Record |
| Cincinnati | 95.6 | 115.7 | 108.6 | 130.7 | 101.5 | 124.0 | 225.6 | 5-2 | 6-1 |
| Rutgers | 92.4 | 106.7 | 125.7 | 97.5 | 107.9 | 98.7 | 206.7 | 2-4 | 4-2 |
| West Virginia | 105.8 | 86.6 | 107.0 | 111.3 | 103.7 | 101.1 | 204.8 | 4-2 | 4-2 |
| Pittsburgh | 108.4 | 106.0 | 104.2 | 84.5 | 109.0 | 93.3 | 202.3 | 4-2 | 4-2 |
| South Florida | 100.6 | 107.0 | 102.5 | 88.1 | 99.6 | 101.1 | 200.8 | 4-2 | 2-4 |
| Connecticut | 95.1 | 97.8 | 85.0 | 130.1 | 91.3 | 108.9 | 200.2 | 3-3 | 3-3 |
| Louisville | 103.2 | 106.1 | 96.7 | 86.0 | 100.2 | 92.2 | 192.4 | 2-4 | 1-5 |
| Syracuse | 97.8 | 82.1 | 61.4 | 95.4 | 83.0 | 88.0 | 171.1 | 1-6 | 1-6 |
Thoughts and observations:
In the end, data at all levels tell a different story, even when you get down to the possession- or play-level data. Just look at North Carolina; the possession-level FEI data thinks very highly of North Carolina's work. At the play-by-play level? Not so much.
One thing play-level analysis misses that possession-level data obviously does not, is the importance of third-down conversions and passing downs situations in keeping drives alive or killing them. That is something that we will begin to explore next week with second- and third-level points.
13 comments, Last at 24 Jul 2009, 7:47am by Hammer
Comments
Re: Varsity Numbers: Conference Breakdowns
This article serves to really irk me that Penn St. is not in the talk of the national championship game at all.
Re: Varsity Numbers: Conference Breakdowns
If I understand correctly, each rating is only compared to the average rating FOR THAT PARTICULAR LEAGUE. Penn State's rating, despite only being the 5th highest rating on the page (and therefore RENDERING Penn State undeserving of national championship game talk because they're an inferior candidate to Florida, Oklahoma, Texas, and USC among 1-loss teams, and inferior to Alabama by virtue of that one loss), also only serves to indicate that they were the best team in what was considered a down league. I suspect if you checked Utah and Boise State's ratings, they'd be through the roof by this measure, but that doesn't make them more deserving of a championship berth once the context of their own conference is stripped away.
The team with a more legitimate gripe about being left out of discussions is USC, in my opinion. They rank as high as Florida and Oklahoma in most of the computer formulas designed to predictively measure team strength instead of simply descriptively measuring the quality of the season to date. Those three teams are clearly, clearly the three hottest (and most likely the three best) teams left standing.
Re: Varsity Numbers: Conference Breakdowns
Correct--the numbers for each conference reflect data for only that conference, so combining a team's Overall "+" from one conference to that of another team in another conference doesn't have a ton of relevance. It probably says something (like "USC was a far more dominant one-loss team in their conference than Penn State or Texas Tech were in theirs), but not a lot. That will come with a more all-encompassing "+" rating after all games have been entered.
Re: Varsity Numbers: Conference Breakdowns
Alabama is the hottest team in the country. They have not lost. I'm a sooner fan and i do not understand this argument.
Re: Varsity Numbers: Conference Breakdowns
"hot" does not mean "longest winning streak". Hot talks about just how well the team is playing at this current point in time, whereas winning streak speaks to how well the team was playing weeks, or even months, ago.
Re: Varsity Numbers: Conference Breakdowns
Bill,
Good article. A couple things I'm not sure about:
1. These are all games for each team, right, not just conference games? With the UT-OU comparison, I'd be interested to see how the two compare both in conference and if you include non-conference games.
2. As noted above, I'm assuming the comparisons are based on conference average, not average across all of DI-A.
3. Given that both OU and TTU, for example, have S&P+ in Close on Offense greater than S&P+ in both overall Rush Off and Pass Off, while Mizzou is the opposite, putting both overall Rush Off and Pass Off doesn't seem to be providing very much valuable information. Relatedly, I also wonder about the balance in Rush Off and Pass Off, and if that's skewing your statistics. USC and Ohio State are two prominent teams that S&P+ is telling us have better Pass Offenses than Rush Offenses. Watching Pryor and Sanchez throw would seem to be a refutation of this observation. Here's the information I think you should have presented: Close Off and Def S&P+, Close Rush Off and Def and Pass Off and Def S&P+, Non-Close Off and Def S&P+ (so we can see what teams are good at blowouts).
4. Maybe you addressed this in a previous article and I'm just forgetting, but games against Alabama and Oklahoma don't go at the same pace-teams normally have fewer plays against Alabama than against Oklahoma. Are your results potentially biased by a team getting 60 plays on offense against the Sooners and 40 against the Tide? (Numbers made up to show the point.) Note I'm not sure how DVOA handles this, either, but it seems like a potential source of bias, and a more severe one in college.
Re: Varsity Numbers: Conference Breakdowns
There's just something wrong with this:
Best Passing Offense: 1) Ohio State
Will
Re: Varsity Numbers: Conference Breakdowns
Best passing offense compared to the rest of the conference NOT the nation. I dont watch much CF but that seems reasonable to me.
Re: Varsity Numbers: Conference Breakdowns
It's really not though. Terrelle Pryor took every meaningful snap during conference play, and his stats, on average, are:
8/13 for 113 yards each game, with 8 TDs and 3 INTs
Any system that makes that the best passing offense is broken. The best passing offense would manage to make 10 completions per game, don't you think? Take out the Penn State game and Pryor didn't even average 100 yards passing per Big Ten game.
Will
Re: Varsity Numbers: Conference Breakdowns
Remember, though, it's a per-play measure. On a per-play basis, Ohio State had the most effective passing game in the Big Ten. Now...chances are, if they threw a lot more, they would become a lot less effective, but the way they use it, it worked alright for them.
Re: Varsity Numbers: Conference Breakdowns
I understand how the measurement here marks them as the best passing offense, but I'm pointing out how ridiculous (or useless) that statement is. To make that kind of statement, you have to have some context. Pryor's numbers look good because:
a) He very rarely threw the ball, so teams would almost always have 8 in the box with others sneaking in to the play as well.
b) In the few passing plays they had, he would rarely throw to a guy unless he was virtually uncovered.
c) He took a lot of sacks, which I imagine count as rushing plays in the metrics.
What we need is a metric that combines effectiveness with volume to truly have a comparison of best passing, best rushing, etc. A baseball analogy would be VORP or WARP I guess.
Will
Re: Varsity Numbers: Conference Breakdowns
Will, if the comparisons made to OPS+ in the introduction of this article are to be believed, then the numbers we are using here are rate statistics, not counting statistics. Therefore it's not important how often Pryor passed the ball, but what happened when he did. I didn't follow Ohio State close enough to be able to comment on Pryor's play, but looking up stats right now both his completion percentage and yard per attempt would rank second in the Big Ten, and by only narrow margins to two different guys. Although he didn't have enough attempts to qualify for the leaderboard, when compared to the rest of the Big Ten it actually makes sense that he would have had the most effective performance.
Re: Varsity Numbers: Conference Breakdowns
For now, I wanted to depict that while it is indeed important for offense to inform defense, the opposite is just as critical merchant account. After all, how is the Red Team supposed to simulate the adversary if it doesn't know how the adversary operates? A good Red Team can exploit a target using methods known to the Red Team. A great Red Team can exploit a target using methods known to the adversary. Therefore, I created an image describing how offense and defense inform each other. This assumes a sufficiently mature, resourced wireless internet providers, and capable set of security teams.I think this is a great idea, but there isn't anything revolutionary about it. It's really just one step above the previous pervasive mindset for digital security, namely identifying vulnerabilities. In fact, this neatly maps into my Digital Situational Awareness ranking. However, if you spend most of your time writing policy and legal documents, and not really having to deal with intrusions, this idea probably looks like a bolt of high speed internet service lightning!
Post new comment