Value Pricing the NFL Quarterback
This was a little something I was kicking around in my head for some time and finally got around to doing. This is a way to attempt to put a composite score to an NFL QB using a number of different efficiency measures and then plotting it against their reported salary, in essence created a Value Mapping of todays NFL QBs. I'll try and explain as best I can here what I did, and if you have any questions on it just drop me a line either here or via email.
Please note that all of the contract numbers used are based on various reports of the contracts yearly value. Since I do not have access to any official records this is the best I can do, though I believe most QB's have their numbers reported very accurately in the media.
In order to qualify for the listing the player had to have 2 qualifying seasons in the last 3 years with a qualifying season being defined as one in which a player has at least 100 pass attempts and participates in 25% of the teams offensive snaps in the same season
There are a number of different stats that attempt to put quantitative scores to a QB performance. Readers of my site know I keep my own stats, which are comparisons to schedule averages, but I maintain those on a team by team basis except for certain Jets where I try to keep them on a player basis. So that makes those pretty much useless here, so I did not attempt to include them in the evaluation.
The most widely know stat is the QB rating (QBR). QBR has been around forever and involves a mathematical formula that comes up with a rating that can reach as high as 158.3. The highest single season QBR belongs to Aaron Rodgers who hit a 122.3 this season. Similar to the stats I keep there is no consideration given to the situation and all results are weighed the same. Unlike my stats there is no consideration given to quality of opposition. More often than not you will hear sports people on TV using the QBR as a definitive mark for a passer.
Defense Adjusted Value Over Average (DVOA) is one of Football Outsiders metrics that measure the value of a number of players, QB included. DVOA is an attempt to place a value on a QB based on situational play. Their baseline is the league average in a situation (down and distance) and then they adjust the success criteria based on the opponent they are playing. Essentially they award a point for a play that picks up a minimum amount of yardage based on down and distance with bonus points awarded for amount of yards thrown beyond the necessary yardage as well as red zone play. They also award negative points for negative plays, such as interceptions. The DVOA then compares the players score to the average expectation. I believe the highest ever DVOA belonged to Peyton Manning in 2004 with a score of 60.6%. FO is often featured on ESPNs website for fans looking for more indepth statistical coverage.
Pro Football Focus has their own grading system, though they have yet to name anything so I will just refer to it as PFF. PFF grades a player on 3 criteria- pass, run, and penalties- and adds them up to come up with an overall grade for a player. Like DVOA it is more of a situational based grading scale with points being awarded between -2 and 2 for a play. They normalize the stats based on an average expectation which they set when they began their site. PFF has caught some steam in the last two years with bloggers and some mainstream reporters. I think the simplicity in their grading will help it gain more traction in the coming years, but for now it is a "pay per view" database which limits their exposure. Drew Brees' 60.2 from 2011 was the best season for a QB that they have recorded.
Percent of Team Yards (POTY) is something I decided to do for this study to give players credit for playing time, staying healthy, and general importance to the offensive scheme in which they have played. For example most of the efficiency stats can be skewed in the favor of players who see the field for a minimum amount of plays or are offensive caretakers. Though their numbers should, in theory, hold up given more opportunities over the course of 16 games that rarely would happen in the real world. There is a reason why the player is on the bench for most of the snaps year after year. That has to be taken into account when putting a value on a QB. For example Joe Flacco grades lower in almost every metric than Mike Vick, but Flacco accounts for about 53% of his teams offense the last 3 years while Vick, who spent one full season on the bench and multiple games hurt, only accounts for 27%. POTY gives the healthy player keeping a job a chance to narrow the gap between those who don't have the opportunity to screw up as often.
For each category I used the stats for the 2009, 2010, and 2011 NFL seasons and averaged the score across those three years. For QBR, DVOA, and PFF if a player did not have a qualifying season I did not count it in the average (i.e. Matt Stafford's 2010 season did not count in the averages due to the 25% requirement). For POTY I do count those seasons in which the player did not qualify. Why? Because injuries or poor practice play need to be factored in. It's not fair for a player like Peyton Manning to not be penalized when Drew Brees never misses a game. The only exception was for the two retired players on the list of Brett Favre and David Garrard and the 2010 rookies. Since the retired guys were no longer in the NFL in 2011 I did not count any stats for their former teams. 2010 rookies did not have any 2009 stats so that would be unfair to penalize them.
Now I did consider doing a weighted average (50% on 2011, 35% 2010 and 15% 2009), but for a first trial run ended up going with this. Admittedly that is probably a stronger way to look at the players, but I had done my POTY across the three years rather than for each individual year when I started to compile data, so this was simpler for me.
I wanted to normalize the scores with a grade of 0 so I did that by taking each stat and taking the lowest score in each category and bringing it either up or down to 0. For DVOA that required adding 28.5 points to ever score to bring Derek Anderson from a -28.5 to a 0. PFF's low man was Mark Sanchez at -19.9. Derek Anderson was again the man, this time for QBR, with a 54, so that led to a subtraction of 54 from each score. Finally Bruce Gradkowski led to a -15.4 point adjustment in the POTY category. This doesn't change the differential between players, it just makes things a little neater to plot.
Each category was given equal weight and the totals simply added up to determine a composite score for the player. In the future I may try something with percentage to average comparisons rather than overall totals.
The top 5 should come as no surprise- Rodgers, Brees, Brady, Rivers, and Roethlisberger. Peyton Manning ranked 6th as he was hit hard by the non play in 2011, but even if we took that season out he was still slightly behind Rivers. In recent years he just has not been as productive as the top guys. To me the most surprising result was David Garrard at 14. He's not great by any means, but a steady performer in each category. Vince Young at 18 was also a surprise, but he plays well in spurts and then heads to the bench.
The real duds are the 2009 and 2010 NFL draft class. Freeman, Bradford, McCoy, Sanchez and Stafford ranked 28, 29, 30, 31, and 32. In Stafford's case he would benefit greatly from a weighted system since he played so much better this season so he may be the guy to justify the contract. The others all have big uphill battles to remain as starters.
Here are the adjusted figures and total score for each player.
Adjusted QB Scoring Matrix
The Value Map
Finally I have a plot of value vs price. I used Excel to plot an exponential fit in there just as a point of reference. This just gives a very rough guideline of players who are likely overvalued or undervalued. Below the line means a team probably got a good deal and above means they are overpaid for production. The contract is the based on the latest contract the player signed.
I split the graph up into 25 point boxes to help compare values better. What you have to do is look up and down each section to see players that have had about the same value over the three year period. For example Sanchez, Bradford, Stafford, McCoy, Cassel, and Shaun Hill have all given about the same on the field aggregate performance. Bradford is horrible value right now. Colt McCoy gives about the same results for a fraction of the price. As you move to the right the value of the player increases so all things equal the more to the right the better the value.
The worst value is Kevin Kolb. His level of play is more or less equivalent to a player who should earn about $3.75 million a year. He makes $12.4 million. He's always hurt and does not play particularly well when healthy. He is Exhibit A of why you stay away from backup QBs that create interest off a handful of games. Matt Cassel is Exhibit B. The success though is Matt Schaub who is tremendous value basically giving you the same results you get from guys who cost around $15 million a season. You can bet that is the guy Matt Flynn will be compared to in free agency when he negotiates his deal with someone.
I still will never understand the Ryan Fitzpatrick signing by the Bills. He ranks right along with all those journeymen making under $5 million a year, which is what Fitzpatrick is, yet the Bills paid him close to $10 million. Drew Brees and Aaron Rodgers right now are the best values out there. It says something about signing guys early and taking a chance. Brees should emerge as the highest paid QB in the NFL since his contract is up, by a few $100,000 more a year than Brady and Manning. There is no way for the Saints to argue otherwise. It just seems like a slam dunk. Rodgers in under contract until 2014. In theory he should be the first $19 million dollar QB, but I can't see the Packers just tearing up the current deal and increasing his salary by that much no matter how good he is. That team is set for the next 3 years because his cap will never exceed $11 million. That's great planning and a great deal of luck too!
Based on the graphing I would think that the players most in danger of being asked to take a paycut next year are Kolb, Sanchez, and Carson Palmer. All are due significant money and carry decent size cap hits. Bradford still has one more year of leeway and Stafford's big 2011 makes him safe. Cassel may be saved because his actual cash payout is low, though Kyle Orton is better value and may choose to stay there and replace him.
I think Jay Cutler and Vick are interesting cases. Both got deals far in excess of performance, Cutler probably moreso than Vick since Vick is dragged down much more by being a backup one season. Cutler's cap is $9.6 million next year and most of his money has already been paid so he is no longer overpaid, its just a firm knock on the deal he signed. Vick is overpaid and will make $12.5 million guaranteed next season. It is going to be a big year for him because his salary jumps to $15.5 million the following season and there is no full guarantee on that money. Right now he only has $3 million guaranteed for injury only. The Eagles are masters at the cap and Vick won't be back at that figure if the same performance occurs.
Again if there are any questions or info you want to pass along drop me an email.
I don't know if there's a statistical tool that does a quantifiable form of clustering, but it might be worth playing around with your data to see what emerges.
Dan- To do something like that I would probably have to cluster the players in groups such as rookie deal, 1st vet contract, 2nd vet contract, and maybe other deals. In terms of fit my guess would be a very poor fit for rookie deal guys (since they were strictly paid based on expectations and draft slotting) with each cluster becoming stronger and stronger as there is more body of work to base the contract on. Actually to probably do this for attempting to see where a contract for a player should be the proper way to do it is to take the 3 years before the players most recent contract rather than the most recent 3 years. Players are paid based on those two and three years leading into a deal. Thats too much work for me to do in my spare time, but I bet would make a pretty decent forecast of future pay.
Jim- Thanks for the link. I hadnt seen that. One of my original iterations with this was to use the yearly cap hit, but the problem with that is that the cap is an accounting measure that is easily manipulated and ties more in with a teams salary structure at that moment in time. The secondary chart is much better, though Im surprised he didnt put salary on the Y axis. I wonder how much the data would change if he pulled rookie contract figures out of it and translated everything to per year? The reason I say that is only because guys not selected top 10 that went on to have good careers are penalized for salary. For example from the time Bradys rookie deal expired until 2009 he accounted for about 67 million in cap dollars. Before that he made nothing. Mannings rookie deal ran through 2003. He accounted for 75.5 million in cap from 2004-2009 but his total is far bigger because his rookie deal added an extra $40 million more or less based just on draft slotting in those 2000-2003 years.
I would not call Stafford a dud, but he was injured a lot his first 2 years, so his 3 year stats are not so good.
(I'll trade Mark Sanchez for Stafford - salary cap hit be damned). I'd also gladly trade Sanchez for Bradford. Bradford may end up not being anything special, but that may be better than Sanchez.
(give Sanchez this year with a new OC to see how he can improve)
Maybe Chad Henne will end up being better than Sanchez. Maybe.
Advanced NFL stats has something similar.
They also rate Brady, Brees, Rodgers as above average
If you sum each column of metrics, you get DVOA with 1327 pts., PFF 886 pts., QBR 1124 pts., and PTOY at 1370 pts. That means that DVOA and PTOY are contributing significantly more to the Total Score of all the QBs than PFF is. I think you should scale each metric so that you get the same number of total points. That way, each metric will be valued similarly in the Total. I doubt it will change things much, but it's important for completeness.
Anyway, interesting idea, and I love this kind of analysis, so keep it up.
Normalizing everything to start from 0 makes sense. Wouldn't it also make sense to map each to the same range? Maybe make each score go from 0 (lowest) to 100 (highest) so after subtracting lowest, you could multiply by 100/(highest-lowest). Otherwise you are giving more weight to the score with the larger range (DVOA in your case).
Thanks for the good read.
Dave and Adam- I tried a few different ways to normalize everything to keep the weights relatively the same. Originally I was going to run with percent of highest possible score but those didnt mesh well with QBR. I had not tried either of the things you mentioned and as soon as I read your comments I pretty much slapped myself in the head for not thinking that. I got so hung up thinking of percentages and such that I just said "forget it let me just add them up and be done"
Anyway I just reran the spread sheets using both criteria. Both made some small changes that are probably a more accurate grouping. Manning jumped Roethlisberger by a bit and Rivers closed the gap to Brady by a small margin. The ranking changes mainly occurred in between with players like Cutler taking a tumble of a few spots and Moore and Smith jumping a few. Stafford also jumps since he was hurt by POTY. None of it would change the graphings much but definitely better rankings. Thanks for the input!
this will be an interesting off season!
While always being cognizant of the quickly escalating contracts on the heels of superb QB prowess on the field, I have never before seen a visual representation of that. I do appreciate the graph.
I wondered, however, what you used as a guide to normalize the figures. I see now that you used the QBRs as a comparison, which makes sense.
I had not considered before, either, that using stats from college games for NFL rookies would be more meaningful considering they are the stats that will be considered for a rookie contract. I am also glad I am not assigned with the task of garnering all of those stats each year.......Whew!
If he improves off his poor 2011 lets say 28-30 tds 11 ints and we win at least one playoff game, that probably bodes well for signing him longterm. Most QBs get a big raise when they sign a new contract, but is it really fair to use Sanchez previous salary as the barometer for a raise? You could make the argument that a good extension could actually be a pay cut from his previous salary.
Great site by the way, love it.