Also, a lot tougher to look into would be whether or not the trade off in regression is worth it.
Ex: with Ehlers I’ve pointed out how even if he were much worse than his xGAR/GAR per 60 with elevated minutes he has a large drop necessary to hit the same level as his alternatives.
So being worse than in the limited role may still mean better than the alternative.
I guess you could look at how analytical darlings did relative to the field of the cohorts they move into.
Yeah, I think there's some selection/survivor bias, but I wasn't sure of the direction of the cause/effect. I would think that if a player is promoted and sees a huge drop in their play, they'd get demoted again quickly, where they may see an improvement in their play again. In that case I think player would show up in the data as not being promoted unless they spent a lot of time in their higher spot. I wasn't sure how to capture that, using GAR though. What I really need is a game-by-game proxy for GAR.
I tried to capture that and cohorts by grouping by line/pairing TOI and color-coding their xGAR/60 by line/pairing level, but I didn't elaborate on it enough. It was striking to me that a little less than half of forwards and about half of defensemen who got promoted to the top line/pair saw their xGAR/60 increase, though that could be the survivor bias issue.
For trade-offs, you're right, that's an important consideration too. When players are demoted they often see a (smaller) increase in their xGAR/60 on net, so a coach would have to balance that with the decrease they're likely to see from the player they're promoting. Plus the effects of chemistry/fit, although I'm not convinced coaches fully understand that either.
Very cool research! I was wondering if for defensemen you did the same as you did with the forwards (with rank and without rank) for the time on the ice vs production. Also is TOI 5v5 or does it even matter in your opinion?
Yes, I did the same rank vs without rank vs scaled by team and got similar results, though it was much closer for defensemen. I used all EV TOI (5v5, 4v4, 3v3) because Evolving-Hockey's xGAR numbers are calculated that way instead of separating each EV strength state. I think just doing 5v5 would change things a little bit, but not very much since 5v5 TOI is something like 98% of all EV TOI.
My only question comes as a result of leveraging. It is easier to put up good rate numbers on a low amount of volume, because in lower minutes, you're still picking up the low hanging fruit. Once significantly more load is put onto you as a player, your minutes will not only be the low hanging fruit anymore, making it significantly harder to play well, which will naturally drop your rate numbers a little bit, even if it increases your volume.
As we've talked about, I'm a complete donkey as it comes to hockey numbers. I actually don't even know the scale of xGAR/60. I don't know what values are good and which are bad, so perhaps this is a silly question, but what brought this to mind in particular was the forward dot that got five and a half more minutes of ice time year over year, but lost 0.25 xGAR per 60 minutes. Is this a regression? Or is this an example of a player's numbers holding fairly well, considering he was moving well out of low hanging fruit territory?
On my side of the tracks, Sam Darnold in the NFL got 506 more plays this year, but endured an EPA/Play drop of -0.031. This is generally conceived not as a regression, but as small sample performance holding up pretty well.
I suppose what I'm asking is, does this leveraging effect exist in this sport? Is there an expectation that somebody's rate numbers will go down with more volume, simply because they've run out of low hanging fruit, the way there is in football and basketball? If that's the case, even some negative year over year changes, depending upon how negative they are, can be construed as positives, like for Sam Darnold in football. If not, I suppose I've read this article from a fundamentally incorrect perspective LOL.
That's a great question. In theory GAR/xGAR account for "easier" minutes as quality of competition/teammates. Moving up the line up players also get to play with better teammates which has a much larger effect on production than competition in the regular season at least. From what I read (years ago, I can't find it now), there wasn't much evidence leveraging exists in hockey to anywhere near the same extent as in basketball or football, just because there's not *that* much of an ice time difference between a first and fourth forward line or first and third defensive pair and the distribution of quality of competition is much narrower than teammates.
I would definitely believe this can make a difference in a single game or playoff series though, where a coach may be chasing certain matchups. The Stars did that this year trying to match Wyatt Johnston against opponents' top lines and he set a record for worst plus/minus (terrible stat and a bit misleading cause of empty net goals against) and generally got caved in terms of possession and expected goals.
As for scale, a change of 0.25 xGAR per 60 minutes comes out to about +/- 4-6 goals over a full season for a player playing top 6/top 4 minutes, so it's not insignificant (almost 1 win over the course of a season). I would chalk that up mostly to 1) players regressing to their mean (coaches do sometimes identify players who should be at the bottom of the lineup) 2) random variation and, 3) systems/teammate fit rather than leveraging.
Most of my hockey watching experience is tied up in my local OHL team, a league where the top forwards can get up to 25 minutes in a game without anybody fussing, and teams routinely play with five and sometimes even four defencemen. I suppose the same logic may not apply to the NHL, where the top ice time forward on my New York Islanders was Bo Harvat's 20 minutes and ten seconds, which I suppose is not hugely different from fourth liner Casey Cizikas's 13:02. It's not like the NFL or NBA, where if a player is a backup to begin with, you can more than double their workload fairly easily.
Funnily enough, this all comes in the same comment where you mention that the biggest factor in the results of an offence is the quality of the offence, not the quality of the opposing defence, which happens to be exactly in line with the NBA and NFL, which just makes my head spin. I can't keep track of it all. Hockey is so unique, but it's also ruthlessly the same as every other North American sport.
Thank you very much again Aaron. Perhaps someday, this donkey will know something about hockey, and I'll place a significant amount of the blame for that on you!
This is great. Do you think there’s some selection bias though in that not all “analytical darlings” are given a larger role?
Also, a lot tougher to look into would be whether or not the trade off in regression is worth it.
Ex: with Ehlers I’ve pointed out how even if he were much worse than his xGAR/GAR per 60 with elevated minutes he has a large drop necessary to hit the same level as his alternatives.
So being worse than in the limited role may still mean better than the alternative.
I guess you could look at how analytical darlings did relative to the field of the cohorts they move into.
Replying to both comments here:
Yeah, I think there's some selection/survivor bias, but I wasn't sure of the direction of the cause/effect. I would think that if a player is promoted and sees a huge drop in their play, they'd get demoted again quickly, where they may see an improvement in their play again. In that case I think player would show up in the data as not being promoted unless they spent a lot of time in their higher spot. I wasn't sure how to capture that, using GAR though. What I really need is a game-by-game proxy for GAR.
I tried to capture that and cohorts by grouping by line/pairing TOI and color-coding their xGAR/60 by line/pairing level, but I didn't elaborate on it enough. It was striking to me that a little less than half of forwards and about half of defensemen who got promoted to the top line/pair saw their xGAR/60 increase, though that could be the survivor bias issue.
For trade-offs, you're right, that's an important consideration too. When players are demoted they often see a (smaller) increase in their xGAR/60 on net, so a coach would have to balance that with the decrease they're likely to see from the player they're promoting. Plus the effects of chemistry/fit, although I'm not convinced coaches fully understand that either.
Such a great piece
Thanks max!
Very cool research! I was wondering if for defensemen you did the same as you did with the forwards (with rank and without rank) for the time on the ice vs production. Also is TOI 5v5 or does it even matter in your opinion?
Thank you!
Yes, I did the same rank vs without rank vs scaled by team and got similar results, though it was much closer for defensemen. I used all EV TOI (5v5, 4v4, 3v3) because Evolving-Hockey's xGAR numbers are calculated that way instead of separating each EV strength state. I think just doing 5v5 would change things a little bit, but not very much since 5v5 TOI is something like 98% of all EV TOI.
Good stuff Aaron!
My only question comes as a result of leveraging. It is easier to put up good rate numbers on a low amount of volume, because in lower minutes, you're still picking up the low hanging fruit. Once significantly more load is put onto you as a player, your minutes will not only be the low hanging fruit anymore, making it significantly harder to play well, which will naturally drop your rate numbers a little bit, even if it increases your volume.
As we've talked about, I'm a complete donkey as it comes to hockey numbers. I actually don't even know the scale of xGAR/60. I don't know what values are good and which are bad, so perhaps this is a silly question, but what brought this to mind in particular was the forward dot that got five and a half more minutes of ice time year over year, but lost 0.25 xGAR per 60 minutes. Is this a regression? Or is this an example of a player's numbers holding fairly well, considering he was moving well out of low hanging fruit territory?
On my side of the tracks, Sam Darnold in the NFL got 506 more plays this year, but endured an EPA/Play drop of -0.031. This is generally conceived not as a regression, but as small sample performance holding up pretty well.
I suppose what I'm asking is, does this leveraging effect exist in this sport? Is there an expectation that somebody's rate numbers will go down with more volume, simply because they've run out of low hanging fruit, the way there is in football and basketball? If that's the case, even some negative year over year changes, depending upon how negative they are, can be construed as positives, like for Sam Darnold in football. If not, I suppose I've read this article from a fundamentally incorrect perspective LOL.
Thanks Robbie!
That's a great question. In theory GAR/xGAR account for "easier" minutes as quality of competition/teammates. Moving up the line up players also get to play with better teammates which has a much larger effect on production than competition in the regular season at least. From what I read (years ago, I can't find it now), there wasn't much evidence leveraging exists in hockey to anywhere near the same extent as in basketball or football, just because there's not *that* much of an ice time difference between a first and fourth forward line or first and third defensive pair and the distribution of quality of competition is much narrower than teammates.
I would definitely believe this can make a difference in a single game or playoff series though, where a coach may be chasing certain matchups. The Stars did that this year trying to match Wyatt Johnston against opponents' top lines and he set a record for worst plus/minus (terrible stat and a bit misleading cause of empty net goals against) and generally got caved in terms of possession and expected goals.
As for scale, a change of 0.25 xGAR per 60 minutes comes out to about +/- 4-6 goals over a full season for a player playing top 6/top 4 minutes, so it's not insignificant (almost 1 win over the course of a season). I would chalk that up mostly to 1) players regressing to their mean (coaches do sometimes identify players who should be at the bottom of the lineup) 2) random variation and, 3) systems/teammate fit rather than leveraging.
I suppose you're right.
Most of my hockey watching experience is tied up in my local OHL team, a league where the top forwards can get up to 25 minutes in a game without anybody fussing, and teams routinely play with five and sometimes even four defencemen. I suppose the same logic may not apply to the NHL, where the top ice time forward on my New York Islanders was Bo Harvat's 20 minutes and ten seconds, which I suppose is not hugely different from fourth liner Casey Cizikas's 13:02. It's not like the NFL or NBA, where if a player is a backup to begin with, you can more than double their workload fairly easily.
Funnily enough, this all comes in the same comment where you mention that the biggest factor in the results of an offence is the quality of the offence, not the quality of the opposing defence, which happens to be exactly in line with the NBA and NFL, which just makes my head spin. I can't keep track of it all. Hockey is so unique, but it's also ruthlessly the same as every other North American sport.
Thank you very much again Aaron. Perhaps someday, this donkey will know something about hockey, and I'll place a significant amount of the blame for that on you!