Is QBR the worst stat ever created?

[kearly]

Wilson is hurt bad by all the sacks in QBR.

And yes, it is a dumb stat.

Kip, my new favorite stat is Toxic Differential. Mostly because it's fun to say.

It dawned on me yesterday that the likely reason that Wilson is perpetually under-rated by QBR is because the stat punishes QBs for getting sacked instead of rewarding them for producing despite poor protection.

I think factoring sacks is pretty stupid. Most sacks are not the QBs fault, and even though I do think Wilson is relatively sack prone, a lot of the sacks he takes would have been interceptions had he forced things. Throwing the ball away isn't always an option, and sometimes taking a sack is the best outcome. And though I've called for Wilson to take intentional groundings on purpose at times, you wouldn't do that every single time you get sacked, because your fumble numbers would skyrocket.

It is fun to read the actual stat heads tiptoe around QBR with passive aggressive "read between the lines" type statements, which to me signals that it was not created by someone with statistical savvy. To use a metaphor, QBR is like the stew a young bachelor makes after he throws every single leftover from the fridge into the pot. Not everything is meant to go together, but that's what QBR tries to do.

 

[amill87]

Squidward vs Jax (99.0 QBR/93.2) 13-21, 199 yards and 0 TD, 63 yards rushing and 1 TD

P Manning vs Balt (83.6 QBR/141.1) 27-42, 462 yards and 7 TDs

lol wut?

Oh I get it, Kaepernick did more to help his team win than Manning because he ran for crucial third downs. It makes sense!

 

[5_Golden_Rings]

Among the QBs with a higher QBR than Russell Wilson right now: Matt Ryan, Colin Kaepernick, and Josh McCown (McCown is #1). Just barely behind Wilson? Cutler, Luck, Vick, FitzPatrick, Jake Locker. According to QBR, Russell Wilson has been just a touch better than either of Tennessee's QBs, and worse than the sum of Chicago's QBs. The 6-5 Chicago Bears have two QBs in the top 10. I'm not joking.

Wilson's not the only QB with a ridiculously low score. Mike Glennon is ranked 22nd and Carson Palmer is ranked 26th.

What is it going to take for this stat to go back to the drawing board?

Cumulative rating is a down side to Total QBR, because all it is is an average of game QBRs. You might have a high QBR in a game where you had an impact on 60% of the plays, and then a bad QBR in a game where you only impacted 40%, but they will carry the same weight in average QBR per game. (Take Colin Kaepernick in his games against the Jags and Seahawks. Against the Jags, Kap contributed a great deal early in the game, the team quickly put it away, and he threw maybe one pass in the second half. Against Seattle, Kap's poor play from start to finish contributed greatly to the Seahawks winning. He had far more "action" plays in the Seattle game, and he played terrible on most of them. And yet, in a Total QBR average, BOTH OF THESE GAMES ARE WEIGHTED THE SAME!)

The QBR tells barley anything about a quarterback. I've noticed that completion percentage is a very big part of QBR. So yes I believe it is a very dumb stat to create and base arguments off of.

No, it isn't. The stat that correlates the most with Total QBR is Adjusted Net Yards Per Pass Attempt, to which completion percentage plays a small part.

http://www.pro-football-reference.com/a ... y.htm#ay/a

This statistic has a greater than 80% correlation with Total QBR, which suggests that this is essentially what Total QBR is minus the weighting for situation and the distribution of credit/blame for a play.


Thus, QBR is about as useful or more as any stat that takes into account comp%, yards/attempt, sacks, TDs and INTs.



But again, the problem comes with season averages. They are NOT weighted per attempt, which can seriously screw up the results.



If these season averages were weighted by attempt, we'd probably have Wilson higher on the list. But they aren't (nor can they be, because each play in a game dynamically effects how important later plays are). His worst game is counted as much as his best game, regardless of whether or not he was actually involved in a higher percentage of plays or not. Total QBR only considers "action" plays, which means you can have an excellent QBR even if you only throw 10 passes, or a horrible one with 10 passes; but whatever you get, it will count EQUALLY as much as the next game, regardless of how much you actually are involved in important plays in that particular game.

Squidward vs Jax (99.0 QBR/93.2) 13-21, 199 yards and 0 TD, 63 yards rushing and 1 TD

P Manning vs Balt (83.6 QBR/141.1) 27-42, 462 yards and 7 TDs

lol wut?

Oh I get it, Kaepernick did more to help his team win than Manning because he ran for crucial third downs. It makes sense!

Yeah, that's actually it; and that Manning threw a bunch of TDs after the game was out of reach or after their chances of winning were already really high based on cumulative statistical data (btw that's the Titans game not the Jax, for Kaepernick).



FIVE of Manning's touchdowns came in the 2nd half. Three came after the Broncos already had a 93% probability of winning the game, and 4 came after they had an 91% probability of winning.


See the win probability curve for the game, from Pro Football Reference:

http://www.pro-football-reference.com/b ... 050den.htm


As you can see, they have a 91.9% chance of winning the game before Manning throws his last four touchdowns.





*I think the key here is how plays immediately affected by the QB directly contributed to expected points, which would indirectly affect win probability. Of course, they aren't giving all the details, so you can't be sure.




EDITED: Let me give you a big example. Look where that probability drops the biggest for the Broncos right before the third quarter. At that particular moment, Manning led the Broncos to a 3 and out while the game was tied at 14 (this included taking a sack on a key third down). That was a hugely important point in the game, and Manning failed to deliver. It brought their win probability down to 41%. That is going to hurt a Total QBR. But later in the game, after that point, Manning began to complete short passes that slowly turned the tide, and after they retook the lead, their chances of winning were pretty high. It was at that point that Manning's contribution to increasing his Total QBR began to have a diminishing return. Every touchdown he threw at that point only raised the win percentage incrementally, because it was already high.

Comprende?








AGAIN, Total QBR does not lend itself well to season averages. It just doesn't work well, for the reasons listed above.

HOWEVER, DVOA from football outsiders is quite useful, I think.

http://www.footballoutsiders.com/stats/qb

Here you can see that Russell Wilson is ranked 8th in the NFL, which I think is quite reasonable. Kaepernick is below him at 12th. Oddly, Nick Foles is #1, but then it's hard to criticize 16 TDs and 0 INTs and 9.6 yards/attempt. Alex Smith is where he should be, at #22. Manning is #2, Brady is #15 (but remember how much he struggled earlier in the year; his numbers will go up). For a season average, DVOA is best in my opinion. Either DOVA or AY/A. However, Total QBR is useful on a game by game basis (but NOT for a season average).



So, in conclusion (tl/dr Total QBR is pretty useful if it is used correctly: to provide insight into SINGLE GAMES ONLY. If you try to use a season average of Total QBR you are going to lose a lot in the translation. The only useful piece of information to get from a season average in Total QBR is that a QB with a high number had a couple of games where he played really well with respect to helping his team win. Not really much else. It says nothing about any general trends, consistency, or the overall picture of how that QB did the entire season.

 

[RolandDeschain]

Rings, many of us are longtime fans of DVOA, just FYI.

 

[5_Golden_Rings]

Rings, many of us are longtime fans of DVOA, just FYI.

I see. Well, it is a hell of a statistic.

But Total QBR isn't useless as long as you keep it in the correct perspective.

 

[MizzouHawkGal]

Rings, many of us are longtime fans of DVOA, just FYI.

I see. Well, it is a hell of a statistic.

But Total QBR isn't useless as long as you keep it in the correct perspective.

Single game sample? That makes sense. Like Roland said I am among those that prefer DVOA.

 

[5_Golden_Rings]

Rings, many of us are longtime fans of DVOA, just FYI.

I see. Well, it is a hell of a statistic.

But Total QBR isn't useless as long as you keep it in the correct perspective.

Single game sample? That makes sense. Like Roland said I am among those that prefer DVOA.

Yeah, pretty much. Total QBR's value is that it can give you some insight into the situational contribution of the QB for a single game. But beyond that there isn't much value to it that can't be found in DVOA and my other personal favorites, adjusted yards per passing attempt and adjusted net yards per attempt, particularly the latter. But if I'm in a pinch, yards/attempt is a solid choice. It carries information about completion percentage as well as how threatening the QB was to the defense. Best of all, these are useful on a cumulative basis as well as a game by game, unlike Total QBR (although QBR can be used to gauge how consistent your QB is if you plot it over games, but still that can be said about any of these stats as well).

DVOA is real gold, though.

 

[RolandDeschain]

DVOA's the only thing I've seen that passes the "informed eye" test 99% of the time. You can sit there and watch some crappy team for a few games and think "hmmm, they kept it close against some opponents that seemed decent", and DVOA will back that up. It'll also backup absolutely putrid teams; and weighted DVOA to see more recent trends really helps you figure out a team you don't follow.

Love Football Outsiders.

I'm also becoming a fan of passer rating differential: http://www.coldhardfootballfacts.com/co ... tats/7893/

 

[SalishHawkFan]

DVOA's the only thing I've seen that passes the "informed eye" test 99% of the time. You can sit there and watch some crappy team for a few games and think "hmmm, they kept it close against some opponents that seemed decent", and DVOA will back that up. It'll also backup absolutely putrid teams; and weighted DVOA to see more recent trends really helps you figure out a team you don't follow.

Love Football Outsiders.

I'm also becoming a fan of passer rating differential: http://www.coldhardfootballfacts.com/co ... tats/7893/

I'd been using that stat myself, but didn't realize someone actually had "discovered" it and was tracking it regularly. Awesome. Love that website. Thanks Roland.

About QBR. Worthless stat even for a game by game analysis. If a guy throws for 400+ yds and 7 TD's, it's going to be a blowout. And if it isn't a blowout, it's because the defense can't stop butter. Is that the QB's fault that the defense can't hold a lead?

In other words: Say Peyton has a 91% chance to win, but his opponent comes back because the DEFENSE SUCKS. Now that chance to win goes down. But Peyton keeps throwing for mass yards and mass TD's. Meanwhile, the defense continues to suck and the other team continues to fight back, so the chance to win keeps sinking. Per QBR, Peyton has a MUCH better day. Exact same game, only the defense sucked.

Reality is, how great Peyton's day was has no bearing on how likely his team was to win. QBR is the most worthless stat ever devised. Not just for the season, but ESPECIALLY on a per game basis.

 

[Subzero717]

Tony Romo has (had going into last week) the highest 4th quarter QBR of any QB. That's all I need to know, to tell me that QBR is like every other football stat. Stupid.

Also hearing Dlifer proclaim Romo elite makes me hate QBR.

 

[themunn]

AGAIN, Total QBR does not lend itself well to season averages. It just doesn't work well, for the reasons listed above.

The problem is - if you're going to weight each game equally, you should weight each pass, each quarter, each touchdown equally.

If you score a TD when you're 30 points down, it should count as much as when you score when you're 30 points up, and if you're 3 points behind or 3 points ahead.

Otherwise, you have to also take into account the actual conditions of the game - a TD in a game against a divisional rival should hold more weight than a TD against the Jaguars.

You can't rank the importance of when you score a TD in a game without ranking the importance of the game too.

The problem I have with both passer rating and QBR is that they're both capped. It shouldn't be possible to have a perfect game - or even a "near perfect" game - as it's always possible to do better. One less incompletion, one more TD, 1 more yard per attempt.

Given two QBs who post these stats:

30 of 40 for 400 yards, 4 TDs, 0 INTs
15 of 20 for 200 yards, 2 TDs, 0 INTs

Passer rating is the same, because all of the ratios are identical - but you'd rather have the guy who threw 4 TDs right?

 

[RolandDeschain]

I'd been using that stat myself, but didn't realize someone actually had "discovered" it and was tracking it regularly. Awesome. Love that website. Thanks Roland.

They've been pimping it for a few years now, I've been looking at it once in a while to see what that stat turned out after the fact from some games I've watched, and it seems pretty good. IMO, you can use passer rating differential + DVOA for a particular game to get a really good idea on how good or bad a team is.

 

[kearly]

QBR's 80% correlation to ANY/A isn't as good as it sounds. Passer rating correlates to ANY/A at a much higher rate than that. Which actually makes sense since QBR factors so many extra things that ANY/A and Passer rating do not.

If they could tweak DVOA to take sacks and drops out of the formula, that would be my preferred metric. I'm just fine with passer rating, even with it's flaws. Even (in the rare scenario) where a QB has a deceptive passer rating, it's usually obvious to everyone. Also, you will NEVER see a QB light things up and finish with a 30 passer rating, which is something that happens every single week with QBR.

Also, I disagree somewhat about QBR as a season average. It's actually a worse stat if you look at individual games. The season average helps iron out the stats inconsistencies, though if what you say is true, the way that it is calculated for the season is incompetently handled.

 

[kearly]

The problem I have with both passer rating and QBR is that they're both capped

30 of 40 for 400 yards, 4 TDs, 0 INTs
15 of 20 for 200 yards, 2 TDs, 0 INTs

Passer rating is the same, because all of the ratios are identical - but you'd rather have the guy who threw 4 TDs right?

Same would be true with ANY/A, DVOA and a lot of other stats that measure efficiency.

Also, I'd prefer the lower of those two lines anyway. If a QB only passed 20 times with a passer rating well over 100, he's much more likely to win the game than a QB who is forced to throw all game long. Wilson has blown teams out with 20 pass attempts on numerous occasions. If you are kicking ass and barely passing the ball, that probably means you are very efficient on your drives. If you are forced to throw for 400, it probably means you have trailed most of the game or have squandered some long drives. Beyond a certain yardage point, win probably actually drops for passers.

Okay, I admit I slightly bullshitted you there. Truth is that they are both great performances. But in terms of efficiency, they are equal, and that's what passer rating measures, efficiency.

 

[JGfromtheNW]

About QBR. Worthless stat even for a game by game analysis. If a guy throws for 400+ yds and 7 TD's, it's going to be a blowout. And if it isn't a blowout, it's because the defense can't stop butter. Is that the QB's fault that the defense can't hold a lead?

In other words: Say Peyton has a 91% chance to win, but his opponent comes back because the DEFENSE SUCKS. Now that chance to win goes down. But Peyton keeps throwing for mass yards and mass TD's. Meanwhile, the defense continues to suck and the other team continues to fight back, so the chance to win keeps sinking. Per QBR, Peyton has a MUCH better day. Exact same game, only the defense sucked.

Reality is, how great Peyton's day was has no bearing on how likely his team was to win. QBR is the most worthless stat ever devised. Not just for the season, but ESPECIALLY on a per game basis.

I agree with this 100% and I think your example of Peyton is spot-on. It's two variables working against each other, one of which relies more on the team than the QB (probability of winning).

I have always thought QBR was just a made up "stat" to blow steam up the ass of whoever is the media darling at the time. It's a lazy way of trying to figure in "clutch-ness" and how good the QB played relative to... the probability of winning?

The best way to measure this is to look at how many game-winning drives/4th quarter comebacks a QB has, imo. It's actually objective, but like QBR, has it's own faults. The only fault (I think) worth mentioning is that a game-winning drive or 4th quarter comeback is inherently a team effort, though QB is the most important position in these situations.

Thank you 5 Rings for the explanation. I agree that QBR is a better "stat" for single game analysis, but it is still too biased for my liking.

In my eyes, going 20/20 for 300 yards and 2 touchdowns is the same PERFORMANCE whether it's in a blowout 35-0 or a close game 35-34. QBR is telling me that the QB's play was "less important" or "less impressive" in the blowout, but realistically, does it even matter? In one game, the QB's own defense was great; in the other game, the QB's own defense gave up 34 points. So QBR tells me how well the QB did relative to how much their own defense sucked?

 

[drrew]

This is why QBR is worthless
C/ATT YDS AVG TD INT SACKS
10/16 164 10.3 1 0 0-0

CAR YDS AVG TD LG
7 54 7.7 2 17

QBR rated this as the 3rd best game by any QB since 2006. We're looking at something like 2000 games in that timespan, and apparently this was the 3rd best performance by a QB in those 2000 games. Sure it is.

 

[Cartire]

Time of possession is the worst stat ever created. QBR is #2.

This guy. :roll:

I actually hate just as much as everyone else, but I did have a conversation via Facebook with Mike Sando last year, and he did tell me to think of QBR more as a probability for success, not an actual rating of a QBs skill. Thus why sacks and weather and 4th qtr points accrue their own part of the, ESPN-eyes-only, formula.

Still think it's dumb.

 

[pmedic920]

When it comes to interceptions.... Occasionally one is as good as a punt but other than that, I'd rather see a sack. In other words, I'd rather our guy take a sack or two, than throw an interception.
QBR only has one use IMHO. I keeps some folks arguing amongst them selves and out of the decent conversations.
Not directed at anybody in particular.
As it is currently formulated... We don't need no stinking QBR.