Something Strange Is Going On with the NHL's Play-By-Play

The Box Scores, They Are A-Changin'

A little over a month ago,

Prashanth Iyer

noticed that a shift was occurring across the NHL: starting in 2023-24 and continuing into this season, teams were having a harder time getting shots on net than any time since the beginning of the RTSS era.

It's harder than ever to get a shot on goal

I had also noticed something similar towards the end of last season as I was developing my own shot model for expected saves: namely that the difference between expected and actual jumped dramatically in 2023-24. The culprit for my worsening xS model, from my initial investigation, was a large increase in the percentage of shot attempts that were marked as hitting a post. According to my findings, the percentage of unblocked attempts labelled as hitting a post increased by roughly 0.2%, from about 1.7% to 1.9% just between the 2022-23 and 2024-25 seasons. This is a much smaller change than the overall change in on net percentage though.

Iyer’s finding sent me back down the rabbit hole on this topic and I now think the shift is being driven by at least two changes within the NHL; one major, and another minor, which I will get into at the end of this article. Before I get into the causes, however, I’ll establish that 2023-24 represented a significant break with prior seasons by looking at both overall shot locations and home/away splits for every team.

All data is as of December 17, 2024.

Shot Locations

First, let’s look at shot density at 5-on-5.¹ For this section, I’m going to focus exclusively on unblocked shots because the NHL only records the location of the shot-blocker, not the shooter, on blocked shots. What I want to know is where teams are taking their shots from, and blocked shots don’t give me that information.

Just looking at the raw density maps from 2019-20 to 2024-25, it’s clear that there has been a change at 5-on-5 play. Starting in 2023-24 teams have been taking a higher share of their shots from the edges of the rink, and fewer from the area between the interior faceoff circle hashmarks. The density changes every year, though, how can we be sure the 2023-24 season was an unusually large change from the prior season?

If we look at the year-to-year change, i.e., the difference derived by subtracting the prior year’s density map from each year, we can compare each season’s change more easily.

Visually, there’s a lot more color in the 2023-24, meaning that there are more locations with a significant change in the density distribution. It is possible that such change is within the bounds of normal. Expanding the range to all seasons since 2010-11 reveals this is not the case though.²

The 2023-24 season has by far the largest year-over-year change of any season in the past 15 years. Another way to compare each season’s change is to sum the absolute value of the difference between the density estimate of each coordinate for a season. A higher value, means more total change from one season to the next.

The sum of the absolute difference for the 2023-24 season is clearly the largest of our 15 season sample. In fact, it’s more than double the next closest season, which is (interestingly) 2022-23, and four times the average change for all other seasons.

There’s one last thing I’d like to check before moving on from shot location. It’s possible that teams are simply taking more shots from further away and that’s what is driving the increase in missed shots. If that is the case, we should expect the missed shot density to have a significant change, while the on net density should be much closer to prior years.

Again, this does not seem to be the case. Instead, there appears to be a systemic shift in all shots along a similar pattern from 2022-23 to 2023-24. Such a large break from previous seasons is not credibly explained only by shifting playing styles and natural year-to-year variation. Something must have changed to the shot location recording process in 2023-24.

Rink Biases

Rink recording bias has been well-established in the NHL for nearly a decade now. Each rink’s scorers count things a little differently (or a lot differently in the New York Rangers’ case), meaning that there is some scorekeeping bias from rink-to-rink in both location and frequency of events. The simplest way to measure this is to compare a team’s home numbers to their away numbers, with the difference between the two being the rink bias.

Looking at distance bias, the 2023-24 season again represents a noticeably large change from prior seasons.

In the first 500 games of the season,³ 2023-24 has the smallest range of biases and the smallest standard deviation. Over the course of a full season, the difference isn’t close. Furthermore, the three seasons with the smallest range and the lowest standard deviation for home rink distance bias are, in order, 2023-24, 2022-23, and 2024-25.

A similar pattern is evident when looking at each team’s percentage of blocked shots, missed shots, and shots on net.

The pattern with recent seasons isn’t quite as strong for blocked shot percentage, but 2023-24 has the lowest variance of any season in the first 500 games. Over the full season, 2023-24 again has an incredibly small range compared to every other season.

By missed shot percentage, 2023-24 has the second lowest variance at the 500 game mark and, again, the lowest over the course of a full season. The best season at the 500 game mark is the 2022-23 season. 2024-25 has the 10th smallest range, but just the 6th largest standard deviation. Over the full season, 2023-24 once again has the smallest variance.

Finally, looking at on net percentage, 2023-24 once again has a much smaller range and standard deviation of rink biases at the 500 game mark than any other season in the sample, first by both metrics, followed by 2024-25, and the smallest over the course of a full season.

For the most part, the three most recent seasons have the smallest rink biases, especially over the course of a full season.

So What’s Causing These Changes?

In my opinion, there are at least two likely causes. One is a shift in playing styles/team-building philosophies.

Sean Shapiro

touched on this in an excellent article from before the 2024 playoffs about Vegas’s defensive zone strategy.

To front or box out? Looking at how NHL teams systematically block shots

Vegas was the defending champion at the time, and the NHL is well-known as a copycat league. Some of the shift in shots away from the hotspot in the slot is almost certainly due to teams emulating Vegas’s box-out strategy. Furthermore, this past offseason several smaller puck-moving defensemen struggled to find new teams in free agency with teams emphasizing bigger bodies on the back end. It’s hard not to see that trend as a response to the Cup-winning Florida Panthers, who play a very heavy forecheck. Larger defensemen, in theory, can clear the slot more effectively.

Second, and likely more impactfully, I believe the NHL has started using the puck-tracking technology behind the NHL EDGE data to supplement, or possibly replace, the play-by-play scorekeepers. I think this change likely started to be rolled out during the 2022-23 season, when shot locations start to shift and home rink bias also starts to shrink. The 2024-25 season has more variability than the previous two seasons, but 500 games is still a small enough sample that I can believe natural variance could be at play.

There may be other factors which I don’t have a way to check using the NHL’s available data, such as different offensive systems and shot selection responding to changing goaltender technique, but I think that the defensive shift and especially the tracking data integration explain the majority of the shift.

Conclusion

So what does this mean? In practical terms not that much, probably. The most recent few seasons’ data is likely a bit more accurate than previous ones, but previous seasons were not wildly inaccurate (again, except for the Rangers). To me, the biggest takeaway is potentially knowing what the natural home/away variance is for a team since the impact of scorekeeper bias is much lower.

The other big impact I see is that expected goal values should not be directly compared from one season to the next. I had already begun to think this anyway, but similar to era-adjusting goals or points, expected goals likely also need to be era-adjusted.

Miscellaneous Other Stuff

Two other things I’d like to mention that don’t really have a place anywhere else. First, I wrote a piece for Defending Big D looking at Jason Robertson’s struggles so far this season. I plan on posting all my Dallas Stars pieces there and keeping this blog for general hockey analysis. If you’re a Stars fan, and haven’t seen it, please check it out!

Second, this Substack just passed 100 subscribers, which is a nice round number. So a huge, huge thank you to everyone who has subscribed and continues to read.

1. For to get shot density, I use the kde2d function from the MASS package in R. I prepare the shot location data by flipping all offensive zone shots to the same side of the rink, then filtering out all shots that aren’t in the offensive zone.

2. I prefer to exclude the 2007-08 and 2008-09 seasons from my analysis because the RTSS system was new those seasons and the quality of the data is noticeably lower than later seasons.

3. There have been 505 games played in 2024-25 so far, but using 505 as the cutoff looks a little weird, so I’m rounding down to a nice, round number.