Back to back 10am games

[hawksfansinceday1]

Unfortunately it won't let me make an account over there since their captcha checker is broken. But if I could sign up, I'd have a lot to say that commenter Ammek. First, he shows a record of games from 1980-1994 that mildly contradicts the findings of the article, but says he only checked 5 of the 7 teams "because he didn't have time." That makes me think the two teams he didn't check had inconvenient findings. Also, going from 1980 to 1994 screams selective sampling. Why not including all seven teams and go from 1980 to 2000 (the article's data begins in 2001). Why leave off 1995-2000? Probably because it had inconvenient data.

Secondly, you can't really "adjust" results for DVOA. As much as I love DVOA, it is unreliable as a stat until pretty late in the year, and by the time it becomes reliable it's basically just an average of the season performance, plus it is weighted towards end of year performance. Case in point why this is flawed, the Seahawks tend to be pretty bad early in the year most seasons, but their final DVOA is #1. When teams ilke the Lions and Cowboys nearly beat a struggling early season Seahawks team, Ammecks formula would treat that game as if Seattle were a 39 DVOA team, when in reality Seattle's level of play at that point in the season was in the negatives. I think its best to just leave DVOA out of it since this is a large enough sample of data for the kinks to work themselves out.

Ammek's comments reek of twisting stats to push an agenda. It pisses me off that I can't sign up over there and point this out.

Don't give up cuz he definitely needs called out. Hopefully they'll repair the captcha.

 

[gaucho]

That little factoid about Olympic world records is going in my back pocket. Have a source you can share on it?

Here it is, page 298, first paragraph. I contacted this guy when I was writing my first paper. Super nice fella from England (not relevant, but wanted to give the old guy a shout)

http://www.entrepatients.net/sites/defa ... reilly.pdf

 

[gaucho]

One more try here. East coast teams playing prime time games ANYWHERE in the country. The games regardless of where they are plated start around 8:30pm est. This is probably to the detriment of the east coast teams since the games are played during what is their nighttime hours.

From your first comment.

I'm curious if a study has been done with east coast teams playing night games and what the results show. I would think playing a game starting at 8:30 est and ending at 11:30pm est favors west coast teams, no?

Sorry for the confusion. Your first comment (highlighted above) asked if those late night games might favor west coast teams so I guess I don't really understand where you are going with the question if you are referring to games not including west coast teams, why would that favor them?

East playing east would be no advantage or disadvantage since both teams are on the same schedule.

East playing at home vs west or even central is somewhat slightly even out due to travel.

East playing against west or central opponents on the road would likely be a huge disadvantage to east coast teams but again this happens probably about 5% as often as west coast teams traveling east and playing 10am games.

In fact the schedule set up by the NFL to help east coast teams prevents any east coast team from traveling more than twice a year to play on the west coast. As mentioned before. [/quote]
[/quote]

It's a really good question, and a strong response. Here's my take, copied and pasted, but really it's just rehash of what was said above (and only deals with east at west prime-time games, as otherwise both teams face the disadvantage, as mentioned above).

Fine, you concede, you win; It's a disadvantage. But are these teams any worse off than East Coast teams that have to play West Coast prime-time games ("night body clock")? In a word, yes. Look back to the core body temperature cycle chart. The difference in body temperature between 10 a.m. and 1 p.m. is far greater than the difference between, say 8 p.m. and 11 p.m. Also, body clocks adjust more quickly to westward travel (92 min/day) than they do to eastward travel (57 min/day). And while there is an increase in home team win percentage (60.7) in night body clock games compared to normal home field advantage (57.5), it is not of the same magnitude as morning body clock games (64.4). And even if this disadvantage was the same (it's not), it is far less common; there are roughly five times as many morning body clock games as night. Put differently, a West Coast team faces the larger morning clock handicap roughly 20 times for every one time the East Coast team faces the lesser night body clock handicap.

 

[gaucho]

Sorry for back to back to back posts, I had lots to respond to.

Thanks for the kind words, guys. I mentioned the topic and the article on a few different message boards and you guys were the best combination of friendly and intelligent.

(However, as a niner fan, I'll never admit as much publicly!)

A couple people posted this on reddit. Some good discussion there too. If you follow the link below and click on "other discussion" you should see 3 different threads on the article (some kinder than others, lol)

https://www.reddit.com/r/Seahawks/dupli ... pst_games/

I also struggled creating an account with the captcha thing. It took several tries but eventually worked.

And yeah, I didn't want to be a dick to that guy, but he's crazy. And the idea of using DVOA to show that the early road losses were expected is absurd. The DVOA is calculated based on those results.

Try this. You and I play checkers 10 times. However, you drink a fifth of gin before we start. I win 9 of 10. It's hard to play checkers shitfaced. But wait, dude argues, maybe it wasn't the liquor. If you look at our checkers power rating, mine is far higher (thanks to the boost that I got by beating you 9 of 10) I should have won, you can't blame the gin.

 

[Popeyejones]

That commenter Ammek. First, he shows a record of games from 1980-1994 that mildly contradicts the findings of the article

It doesn't mildly contradict it. It directly contradicts it, showing that there's no difference at all when looking at it one way, and actually a "benefit" to playing early road games on the east coast when compared to traveling to central time zones.

but says he only checked 5 of the 7 teams "because he didn't have time."

Yeah, noticed that too. If you have doubts about it why not look into it?

Also, going from 1980 to 1994 screams selective sampling.

This is only true if you think Brian's 15 year sampling also screams selective sampling. That guy's point wasn't that the data presented were wrong, his point was that 15 years might not be enough data, as depending on the 15 year period you select, you can flip the association.

Secondly, you can't really "adjust" results for DVOA. As much as I love DVOA, it is unreliable as a stat until pretty late in the year, and by the time it becomes reliable it's basically just an average of the season performance, plus it is weighted towards end of year performance.

Yeah, that's something he acknowledged in the comments section. Do you think end of year DVOA is a better or worse measure than straight up wins and losses? Either way, over the time period he's looking at, the data point in the same direction.

It's imperfect data for sure, but points in the same direction as the more clear-cut descriptive statistics, so it's at least a second data point in the same direction, while obviously not nailing up any coffins (the way statistics tends to work in practice, rather than ideals).

Ammek's comments reek of twisting stats to push an agenda. It pisses me off that I can't sign up over there and point this out.

Many, many of the commenters over there are saying the same thing about Brian's post. They're pointing to the same thing I had pointed to (the deceptive fallacy of truncating the Y to signal significance), in addition to actually doing the work to show that these association might not hold, and might just be an artifact of the years selected.

As far as pushing an agenda goes, the people over there, who are statistically literate, see sentences like the two cut-and-pasted below and shudder at the agenda pushing in it:

We again see the expected increase in pass drop rate among teams playing morning body clock games (5.41 percent compared to the 4.97 percent rate of all other visiting teams), but this time not conclusively (P = 0.0719). The relatively small sample size and factors such as prevent defenses employed late in games by teams with large leads may have masked this correlation.

You don't get to futz with statistical significance as you go in order to make your case. Once you accept the basic parameters of what you're doing you have to stick with them, like it or not.


EDIT: To be clear I'm not knocking Brian here. He's responding to the comments over there exactly as he should be, and doing a great job at it too. :th2thumbs:

 

[formido]

That commenter Ammek. First, he shows a record of games from 1980-1994 that mildly contradicts the findings of the article

It doesn't mildly contradict it. It directly contradicts it, showing that there's no difference at all when looking at it one way, and actually a "benefit" to playing early road games on the east coast when compared to traveling to central time zones.

but says he only checked 5 of the 7 teams "because he didn't have time."

Yeah, noticed that too. If you have doubts about it why not look into it?

Also, going from 1980 to 1994 screams selective sampling.

This is only true if you think Brian's 15 year sampling also screams selective sampling. That guy's point wasn't that the data presented were wrong, his point was that 15 years might not be enough data, as depending on the 15 year period you select, you can flip the association.

Secondly, you can't really "adjust" results for DVOA. As much as I love DVOA, it is unreliable as a stat until pretty late in the year, and by the time it becomes reliable it's basically just an average of the season performance, plus it is weighted towards end of year performance.

Yeah, that's something he acknowledged in the comments section. Do you think end of year DVOA is a better or worse measure than straight up wins and losses? Either way, over the time period he's looking at, the data point in the same direction.

It's imperfect data for sure, but points in the same direction as the more clear-cut descriptive statistics, so it's at least a second data point in the same direction, while obviously not nailing up any coffins (the way statistics tends to work in practice, rather than ideals).

Ammek's comments reek of twisting stats to push an agenda. It pisses me off that I can't sign up over there and point this out.

Many, many of the commenters over there are saying the same thing about Brian's post. They're pointing to the same thing I had pointed to (the deceptive fallacy of truncating the Y to signal significance), in addition to actually doing the work to show that these association might not hold, and might just be an artifact of the years selected.

As far as pushing an agenda goes, the people over there, who are statistically literate, see sentences like the two cut-and-pasted below and shudder at the agenda pushing in it:

We again see the expected increase in pass drop rate among teams playing morning body clock games (5.41 percent compared to the 4.97 percent rate of all other visiting teams), but this time not conclusively (P = 0.0719). The relatively small sample size and factors such as prevent defenses employed late in games by teams with large leads may have masked this correlation.

You don't get to futz with statistical significance as you go in order to make your case. Once you accept the basic parameters of what you're doing you have to stick with them, like it or not.


EDIT: To be clear I'm not knocking Brian here. He's responding to the comments over there exactly as he should be, and doing a great job at it too. :th2thumbs:

Do you think body clock issues affect west coast teams traveling east or no?

 

[gaucho]

That commenter Ammek. First, he shows a record of games from 1980-1994 that mildly contradicts the findings of the article

It doesn't mildly contradict it. It directly contradicts it, showing that there's no difference at all when looking at it one way, and actually a "benefit" to playing early road games on the east coast when compared to traveling to central time zones.

but says he only checked 5 of the 7 teams "because he didn't have time."

Yeah, noticed that too. If you have doubts about it why not look into it?

Also, going from 1980 to 1994 screams selective sampling.

This is only true if you think Brian's 15 year sampling also screams selective sampling. That guy's point wasn't that the data presented were wrong, his point was that 15 years might not be enough data, as depending on the 15 year period you select, you can flip the association.

Secondly, you can't really "adjust" results for DVOA. As much as I love DVOA, it is unreliable as a stat until pretty late in the year, and by the time it becomes reliable it's basically just an average of the season performance, plus it is weighted towards end of year performance.

Yeah, that's something he acknowledged in the comments section. Do you think end of year DVOA is a better or worse measure than straight up wins and losses? Either way, over the time period he's looking at, the data point in the same direction.

It's imperfect data for sure, but points in the same direction as the more clear-cut descriptive statistics, so it's at least a second data point in the same direction, while obviously not nailing up any coffins (the way statistics tends to work in practice, rather than ideals).

Ammek's comments reek of twisting stats to push an agenda. It pisses me off that I can't sign up over there and point this out.

Many, many of the commenters over there are saying the same thing about Brian's post. They're pointing to the same thing I had pointed to (the deceptive fallacy of truncating the Y to signal significance), in addition to actually doing the work to show that these association might not hold, and might just be an artifact of the years selected.

As far as pushing an agenda goes, the people over there, who are statistically literate, see sentences like the two cut-and-pasted below and shudder at the agenda pushing in it:

We again see the expected increase in pass drop rate among teams playing morning body clock games (5.41 percent compared to the 4.97 percent rate of all other visiting teams), but this time not conclusively (P = 0.0719). The relatively small sample size and factors such as prevent defenses employed late in games by teams with large leads may have masked this correlation.

You don't get to futz with statistical significance as you go in order to make your case. Once you accept the basic parameters of what you're doing you have to stick with them, like it or not.


EDIT: To be clear I'm not knocking Brian here. He's responding to the comments over there exactly as he should be, and doing a great job at it too. :th2thumbs:

Good stuff! A few comments:

- The body clock effect is real. Therefore, there's no way the guy can "disprove" it by looking at 15 different years. I haven't looked at his data (I will) but really don't need to, the effect will be there. It's possible that in that era with no salary cap and thus more sustained success for given teams, that there is a correlation between location and win percentage. That's easy to control for, and once I do we will see the effect. I promise.

- I like his idea of using something other than record to measure "expected" win percentage, but year end DVOA numbers is laughable. Those numbers are based on what happened that year. You can't go back and say that based on these year end numbers, there is no body clock effect (i.e. the team that "should have won" did) because you are basing your expectation in those games on the results of those games. Ass backwards.

- I'm secretly super amused by the axis truncation thing. I think someone on here called it people wanting to flaunt the size of their statistical dong (which is hilarious!). A bunch of people (not you, Popeye) took a first year stats class, or read a famous book from the 50's and want to discredit my results based on one "fact" that they remember. A few comments here:
I don't think anyone on any forum has actually misinterpreted the data based on the graph. Instead, everyone is just scared that someone else will (but nobody does).
There was zero mention of this when these same graphs that were published in the Journal of Exercise Physiology in February. This national journal has a board of scientists that approve the paper before it is published.
There was zero mention of this when I worked directly with Aaron Schatz tweaking the article, nor by anyone else at FO.
It is not a problem.
And why do the axis-gate folks not have a problem with the temperature graph? Isn't that also truncated? Should I show body temperature values down to 0 degrees F? Isn't that really a truncation as 0 degree F is an arbitrary 0. Should I use Kelvins? The whole thing is ridiculous.

- Finally (I hope this doesn't read as defensive I really do appreciate the comments) If I wanted to futz with the data for pass drops, I would just change alpha to 10% for the entire article and bam, pass drops are statistically significant. I did not do this. Does body clock effect pass drops? Hell yes it does. How could it not (based on the physiological detriments)? Give me 100 WR's and have them run controlled tests in a lab in the a.m. and p.m. and the results will be clear. However in a game, things like prevent defenses mask this correlation. That's all I'm saying.

 

[Popeyejones]

Do you think body clock issues affect west coast teams traveling east or no?

I'd characterize myself as a skeptic who hasn't spent enough (really any) time looking into it (as have Brian and that guy in the comments section) to warrant having an informed opinion on it.

In this thread I've definitely suggested that Brian should be worried about his prior (i.e. A belief in a meaningful causal relationship) and that it might be creeping into his analysis, but to be fair, of course I'd say this, because my prior is the null (if making a guess I would say that no, my guess is that I don't think it affects west coast teams, and if it does, not to the degree that makes it worth really talking about).

It's super tough to budge someone off their prior, and I still think that if I spent some time on it I could "break" Brian's model (as that commenter was doing). I haven't done that though, nor is it Brian's job to do all of my work for me a should I actually want to have an informed opinion.

This is a long way to say I personally doubt it, but I wouldn't listen to me over Brian either.

Although I still have my doubts, he definitely made me consider something, and open to the possibility of it, that I probably would have otherwise just dismissed out of hand.

 

[Popeyejones]

Good stuff! A few comments:

1) The body clock effect is real. Therefore, there's no way the guy can "disprove" it by looking at 15 different years. I haven't looked at his data (I will) but really don't need to, the effect will be there. It's possible that in that era with no salary cap and thus more sustained success for given teams, that there is a correlation between location and win percentage. That's easy to control for, and once I do we will see the effect. I promise.

2) I like his idea of using something other than record to measure "expected" win percentage, but year end DVOA numbers is laughable. Those numbers are based on what happened that year. You can't go back and say that based on these year end numbers, there is no body clock effect (i.e. the team that "should have won" did) because you are basing your expectation in those games on the results of those games. Ass backwards.

3) I'm secretly super amused by the axis truncation thing. I think someone on here called it people wanting to flaunt the size of their statistical dong (which is hilarious!). A bunch of people (not you, Popeye) took a first year stats class, or read a famous book from the 50's and want to discredit my results based on one "fact" that they remember. A few comments here:
I don't think anyone on any forum has actually misinterpreted the data based on the graph. Instead, everyone is just scared that someone else will (but nobody does).
There was zero mention of this when these same graphs that were published in the Journal of Exercise Physiology in February. This national journal has a board of scientists that approve the paper before it is published.
There was zero mention of this when I worked directly with Aaron Schatz tweaking the article, nor by anyone else at FO.
It is not a problem.
And why do the axis-gate folks not have a problem with the temperature graph? Isn't that also truncated? Should I show body temperature values down to 0 degrees F? Isn't that really a truncation as 0 degree F is an arbitrary 0. Should I use Kelvins? The whole thing is ridiculous.

4) Finally (I hope this doesn't read as defensive I really do appreciate the comments) If I wanted to futz with the data for pass drops, I would just change alpha to 10% for the entire article and bam, pass drops are statistically significant. I did not do this. Does body clock effect pass drops? Hell yes it does. How could it not (based on the physiological detriments)? Give me 100 WR's and have them run controlled tests in a lab in the a.m. and p.m. and the results will be clear. However in a game, things like prevent defenses mask this correlation. That's all I'm saying.

:th2thumbs:

Numbered your points to make responss more clear.

1) Yeah, I don't think anyone is claiming that the body clock effect isn't real (or knows enough about it to make a substantive claim). What the guy is claiming is that if it's real or not a meaningful effect from it may not show up in NFL games. I kind of touched on this earlier in my discussion of a hypothetical huge N and the problem big data people run into: statistically significant coefficients of meaningless effect size.

Point taken re: the salary cap era, but I think you should apply the same level of skepticism that you do to null findings as you do to non-null findings. Did the salary cap change the shape of the W/L distribution in any given season, or just how long particular teams stayed at either tail across years? Another counter hypothesis might be that body clock stuff should matter more in prior eras when folks like P.C. weren't adjusting for them, but instead we see the reverse (i.e. welcome to noise ).

2) Agreed that end-of-year DVOA poses an endogeneity problem. I don't think it should do THAT much to your models though, as we're talking a pretty small effect on a few games in a season. The question is if end-of-year DVOA is better or worse than record, and I'm not too sure there's a right answer to that without looking into it. I don't think week-by-week DVOA is a good solution however, as we know that week-by-week DVOA is noisy as hell, particularly at the beginning of the season.

3) I'd be annoyed by axis-gate ( :lol: ) if I were you too, but remember, you've now heard the same thing from many different independent people.

Re: the y on body temperature, the axis is fine as it covers the normal range of human body temperatures (to start the y down at zero would be to suggest that x also applies to a dead person in the snow in Alaska :lol: ). What the commenters over there are saying is the same thing I said before: in the NFL the record range is most years goes from about 2-14 to 14-2, so unless your Y goes from 0-1 its going to make your effects look bigger than they are.

It being a "problem" is of course subjective, but if other statistically literate people are suggesting it's a problem, I think it is a problem, as you are needlessly introducing doubt among the population of people who are must equipped to be convinced.

Re: Nobody misinterpreting it, I think this is a poor argument, as one would have to knowingly misinterpret and still not misinterpret it at the same time. What people are saying, myself included, is that they misinterpreted the effect size until they realized the y was so dramatically truncated, and then they felt deceived.

4) Yeah, you could just engage in a post-hoc change of your alpha, but it would introduce the same doubt as your N does n't seem to be anywhere near small enough to justify that, and the effect size is crazy small.

If (just rough estimations for easy math) you assume 25 completions per game and 4 morning east coast road games per year the effect you've found is that once every other year a pass will be dropped.

Or, put another way, pretending the effect was statistically significant (which it wasn't), you've found that morning AM games dropped Wilson's completion percentage from last year from .6819 to .6812, and we don't even report completion percentage to the 10 thousandths column.

Re: P, the issue is only that you try to excuse away a non-significant alpha at .072 right after merrily accepting an alpha at .044 for turnovers. If you don't want to accept .05 as a dichotomous value I totally get that (i.e. the difference between sig and insig is not itself sig), but the range of acceptability needs to extend both directions from .05 unless you want to introduce doubt about your impartiality. TBF people do this all the time (the "nearly" or "close" or "marginally" significant shuffle), but they really shouldn't, and many more don't.

 

[gaucho]

As always, you make great points. A few quick responses, as trying to keep up on this on various teams sites, FO and the reddit pages has left me a little fatigued.

The axis thing - agree to disagree? I can't believe I've discussed it as much as I have. Hearing well thought out arguments like yours means way more to me than the fact that several people have taken issue with it. The nature of internet message boards is people trying to sound smart by disagreeing needlessly. As some meme pointed out, if you want a question answered immediately online, don't ask it, but instead state the wrong answer.

My closing argument - the axes are labeled, and the relative differences are stated in the text. That should be sufficient. The fact that none of my colleagues in academia, nobody on the review board at the American Society of Exercise Physiologists and none of the Football Outside editors (including the editor-in-chief who I worked with directly) made any mention if this leads me to believe that at worst it's a matter of preference.


The pass drop thing - FO requested specifically that I include that as win rate had already been discussed but measures like penalties, turnovers and pass drop had not. In my JEP article I said that there was an increase in the pass drop rate (citing the percentages) but that it wasn't pronounced enough to be statistically significant. I suggested that factors like prevent defenses could mask a possible effect and that it is left as a topic of future study. I stand by all of that. I think it is relevant that the pass drop rate increases, but important to point out that it's not a large enough increase to be statistically significant.


Back to the important stuff. You're 100% right that the very nature of this problem is hard to measure. It would not be at all surprising to hear of significant competitive changes that get washed out when looking at game results. Given this, the fact the home teams have won 65% of their games when the visitor was in this situation is astounding.

The physiology predicts an advantage, and the game results show this advantage. The NFL has made similar changes in the past at the behest of east coast teams, so why not appease the west coast teams to whom this is a huge deal? This is rhetorical, we all know the reason is because they prefer little competition to their national game.

The fact that the NFL is choosing these ratings over competitive integrity is something that needs to be brought to everyone's attention.

 

[Popeyejones]

^^^^ :th2thumbs:

We're just going to have to seriously disagree on your Y truncation, though.

Even in a long article arguing FOR truncating the Y making sense sometimes, quartz ends on this point:

Always use a zeroed y-axis with column and bar charts.

Of course column and bar charts should always have zeroed axes, since that is the only way for the visualization to accurately represent the data. Bar and column charts rely on bars that stretch to zero to accurately mirror the ratios between data points. Truncating the axis breaks the relationship between the size of the rectangle and the value of the data. There is no debating this one.

They're not the end all be all be a long shot, but they can't even argue for truncating the y without saying that you shouldn't be doing it for precisely the same reason as I've been saying, and others in the comments section there said.

http://qz.com/418083/its-ok-not-to-star ... s-at-zero/

(note: they also use body temp as a good case for truncating).

To be clear I'm not suggesting you go back and change it or anything, but rather, suggesting it's something to be mindful of in the future. :th2thumbs:

 

[gaucho]

^^^^ :th2thumbs:

We're just going to have to seriously disagree on your Y truncation, though.

Even in a long article arguing FOR truncating the Y making sense sometimes, quartz ends on this point:

Always use a zeroed y-axis with column and bar charts.

Of course column and bar charts should always have zeroed axes, since that is the only way for the visualization to accurately represent the data. Bar and column charts rely on bars that stretch to zero to accurately mirror the ratios between data points. Truncating the axis breaks the relationship between the size of the rectangle and the value of the data. There is no debating this one.

They're not the end all be all be a long shot, but they can't even argue for truncating the y without saying that you shouldn't be doing it for precisely the same reason as I've been saying, and others in the comments section there said.

http://qz.com/418083/its-ok-not-to-star ... s-at-zero/

(note: they also use body temp as a good case for truncating).

To be clear I'm not suggesting you go back and change it or anything, but rather, suggesting it's something to be mindful of in the future. :th2thumbs:

Fair enough. Thanks!

 

[Laloosh]

Nerds!