The Current QB Rating Formula Sucks

I did a little research this weekend.  I looked at the calculation for the current QB rating system and, you know what?  It kind of sucks. 

Explanation of the Formula

Here’s the formula – it’s based on 4 general categories.  The specific category for each part of the calculation is in the brackets:

a = ((Comp/Att) * 100) -30) / 20  [Completion percentage]

b = ((TDs/Att) * 100) / 5  [Touchdown Pass pct]

c = (9.5 – (Int/Att) * 100))/4  [Interception pct]

d = ((Yards/Att) – 3) / 4  [Yards per attempt]

The final formula is (a + b + c + d)/.06

a, b, c and d can not be greater than 2.375 or less than zero.

[source:  www.primecomputing.com]

So that you don’t have to go too nuts digging into what the formula is doing, I’ll try to explain as best I can.

The intent of the formula is to give, essentially, equal weighting to each of these categories.  Since some of the numbers are percentages and others are integers, some data manipulation needs to take place (e.g., multiply the percentages by 100). 

Also, not all stats are on the same scale – even if they are both percentages.  For example, 10% is a very good touchdown percentage, but it is a horrible completion percentage.  So, each of the categories is divided by a different amount.  There are a few other pieces in place to “normalize” the numbers and try to ensure equal weighting.

The Formula Applied

To finish up the explanation of the calculation, let me give an example and show you how the categories factor into the final rating.

A player with a 50% completion percentage, 5% TDs, 5.5% interception percentage, and 7 yards per attempt will have a QB rating of 66.67.

This is how each category contributed to the rating:

Completion Percentage = 16.67

Touchdown Percentage + 16.67

Interception Percentage + 16.67

Yards Per Attempt + 16.67

Final QB Rating = 66.67.

So maybe I cheated a little with the numbers chosen.  But, to me, it appears that, when the formula was written, the statistics I laid out above (50% comp, 5% TD, 5.5% Int, 7 Y/A) is what the NFL considered an average or midpoint quarterback.  So, a rating of 66.67 was considered the midpoint of the rating.

I know you’re saying that, today, a rating of 67 gets you a one way ticket to the CFL.  But that’s likely a result of changes in the game – including more restrictions on defensive backs and the growth of efficiency oriented (for lack of a better term) pass offenses like the west coast offense.  Some of these offenses actually replace elements of the running game (for an example see:  Patriots, New England).  This leads to significantly higher completion percentages and fewer interceptions.

Why it Sucks

After reviewing the calculation, I came to the following conclusion.  It sucks.  I have a few reasons why:

·             The formula doesn’t give enough weight to yards per attempt.

·             The formula gives far too much weight to completion and TD %

Despite looking like a very fancy calculation, there is no real science behind it.  I understand that nobody wants to overcomplicate things.  But, if you’re going to have something that looks as horrible as this formula (thereby making things look complicated), there should be some rationale behind it.

I have run a regression model and found that yards per attempt is by far the biggest driver of points scored.  (By no means do I claim to be the first to discover this.  Just check out ColdHardFootballFacts.com or advancednflstats.com to see some pretty in-depth analysis on this topic.  Or you could stay on nflstatanalysis.com, which also has some brilliant stuff – just saying). 

At the end of the day, a quarterback’s job is to score points.  The model also suggested that interception percentage plays a role, but not nearly as great as Y/A.

As far as the other two categories?  Let’s address them in order:

Touchdown percentage, in my mind, is what Brian Burke at AdvancedNFLStats.com would call an “intermediate outcome” (my apologies if I’m mis-using the term).  This means that a player who’s good at other things (e.g. moving the ball down the field) is likely to get more TD opportunities and, therefore, more TD passes.  While better quarterbacks tend to throw more TD passes, it’s probably because the quarterback has more yards and completions and really, a TD pass is really just a completion on a different part of the field.  With that said, I don’t have a huge issue with TD % being a part of the calculation. I just don’t think it adds much.

However…….

I have several issues with completion percentage being a part of model.  Evidence suggests that completion percentage, in and of itself, has no real bearing on points or outcome.  So, it doesn’t deserve equal billing with the other categories – especially Y/A. 

But, here’s the thing.  Although it's not immediately apparent, completion percentage actually has a higher weight than each of the other factors. 

Think about it.  On the surface, completion percentage and Y/A (as well as Int and TD %) get equal billing in the formula.  But, the formula to calculate Y/A could be restated as yards per completion times completion percentage.  This means that completion percentage actually accounts for somewhere around one-third of the model…….and, it doesn’t add anything that Y/A doesn’t.

This would be the equivalent of including yards per completion as well as Y/A.  The two metrics are, essentially, functions of each other.  And, like completion percentage, a player could have averaged 40 yards per completion – but, if he only completed 10% of his throws, he didn’t have a very good game.

Yards per attempt is all you need.

The Extreme Example of Why the QB Rating Sucks

Take the examples below – and yes, I’m intentionally using extremes.

Quarterback 1 – let’s call him Chad Mennington – dinks and dunks his way through a Sunday afternoon.  He finishes 30 for 40 for 100 yards and 1 TD with 0 interceptions.  I don’t know whether he won the game.  And, frankly, it doesn’t matter as anyone watching on TV or in person has long since killed themselves out of boredom.

Quarterback 2 – let’s call him Jay Cutler….I’m not even trying anymore – completes 10 passes on 40 attempts for 350 yards and 2 TDs with 0 interceptions.

What would their QB ratings be?  Quarterback 1 would have a rating of 85.4 while quarterback 2 would have a rating of 80.2.  (See Table 1 to the right)

 

Quarterback 1 probably did nothing to move his team down the field.  But, because his harmless passes were not incomplete, he is given a high rating.

What Next?

I’m just some schlep who is using a geeky website as a way to work through an early mid-life crisis – so, I’ll never have any impact on the NFL changing their QB rating calculation.

But, I have taken a stab at it anyway.  You could actually rank QBs simply by their Y/A and get a better indication of a player relative worth.  Again, go to ColdHardFootballFacts.com to see someone do just that.

I’ve taken it a step further and calculated a rating based on Y/A and interception percentage – with a much higher weight being placed on Y/A (per the regression model).  Additionally, the actual metric used was not simply Y/A, but Y/A adjusted for the quality of the opponent.  The metric was Y/A minus the average Y/A of the opponent (the same metric was used for interception percentage).  My adjusted QB rating is currently called (Opponent Adjusted QB Rating – or OA QB Rating – or OAQBR – but I’ll think of something better) See the results here. 

Most of the ratings were in line between my rankings and the QB rating system.  But, in my mind, OAQBR adjusted certain anomalies that gave too much credit to completion percentage (and, to a lesser degree, TD %).  Some examples:

Philip Rivers

After week 3 (not including Monday night’s game), Philip Rivers ranked 18^th in the league with a rating of 86.1.  He had an OABQR of 6.36 – good for 2^nd in the league.

What drove the difference?  Rivers is has averaged almost 9 yards per attempt – which gave him a good score in my ratings.  However, his 59% completions brought him down in the traditional ratings system.  (His TD % is not exceptional – this statistic is not part of my calculation).

With that said, Rivers has his team at 2-1 and averaging 24 points per game.  He is also leading the league with 991 yards passing. 

Essentially, Rivers is being penalized for taking shots down the field.  His production (in terms of yards) is higher than any other quarterback.

The system favors the dink/dunkers.

Brett Favre

This week's great finish aside, Brett Favre has done very little to help his team score points.  He's averaging just over 6 yards per attempt - which is not even in the top 20.  But, he's completing 70% of his passes - albeit for almost no yardage - so he has a qb rating of 94.5.  This makes him the 8th highest rated passer in football.

In my ratings, Favre ranks 26th with a -3.39.  That's not to say he sucks or hasn't played well.  He just hasn't done anything to merit a lofty rating at the game's most important position when all he's done is play pitch and catch with a couple of guys 5 yards away from him.