How Do Coaching Changes Impact Players?

Image via wikimedia commons

The Vancouver Canucks will have a new coach for the 2013-14 season. While we don’t know who yet, we know it won’t be Alain Vigneault. While the power-play is probably going to be target number one for a new bench boss, and while general manager Mike Gillis is sure to make some changes to the roster in the interim, I wanted to get a feel for how coaching changes can effect individual players.

In order to do so, I pulled all of the wonderful Behind the Net data from the 2007-08 season through to this past season. I wanted to test how consistent statistics like Corsi (a possession indicator based on total shot attempts for and against), Corsi QoC (the quality of competition a player faces), Corsi QoT (the quality of teammates a player plays with), and OffZone% (the percentage of shifts a player starts in the offensive zone) are year-to-year. Once we know that, we can compare the entire player pool against players who underwent coaching changes.

Read past the jump for more.

There are some difficulties with this methodology. First, to make sure a season is appreciable, I had to filter out all seasons where a player played less than 20 games. I also had to filter out seasons where a player changed teams during the year or in the following year – while this would show a change in usage perhaps related to the coaches, there are a lot of other things going on here. Finally, player usage and performance can change based on many factors not related to the coach, but the hope is that with enough of a sample we’ll see some sort of difference.

These qualifications give us 1970 instances where a player played back-to-back 20-game seasons with the same team. The first thing I wanted to look at was the year-to-year correlation of the four advanced stats mentioned above. This will be our "baseline" and basically tell us "this is how much of Stat X can be predicted just by looking at iut’s previous year value." I also threw in PDO (to show that it’s not consistent) and time on ice per 60-minutes (out of curiosity).

So if you’ve ever wondered why people claim PDO (a measure of on-ice shooting percentage and save percentage) is said to regress heavily, there’s your reason why – it’s wildly inconsistent year to year. Ice time, meanwhile, is the most consistent, which perhaps shouldn’t be surprising. More than three-quarters of a player’s ice-time can be explained just by looking at his ice time from the previous year.

But with the four stats we’re focused on, we see that they’re not all that consistent, ranging from an R-squared of .32 to .37. This means that for our Corsi-based stats aren’t very predictive, with roughly one-third of their value being determinable by the prior year’s value. (As a reference point, we can predict about 40 percent of Corsi if we know a player’s Corsi Rel QoC, Corsi Rel QoT and OffZone%.)

But the original question I asked pertained to coaching changes, so we want to separate these seasons into seasons where players stayed on the same team but the coach changed. (Note that if a coach changed close enough to the start or end of the season, I considered it an off-season coaching change.)

This limits our sample now to 572 player season-pairs where a coaching change occurred in the off-season while the player stayed on the same team and 1398 player season-pairs where the coach and player remained in place.

Now we’re seeing something significant. Not unexpectedly, new coaches tend to shake things up. A decrease in QoT stability points to line tinkering; a decrease in QoC stability points to changing roles; a decrease in ice time stability suggests that perhaps new coaches have less  allegiance to certain players. The stats are consistently more stable or predictable when the coach stays the same.

Of course, a coaching change likely also indicates that a team did poorly, which probably means that there were other shake-ups to the team that may cause changes. Still, the difference is consistent across the board, and it means that the Canucks who stick around for next season are more likely than they would have been otherwise to see their roles and responsibilities shaken up.