Adjusted Plus-Minus: Measuring Defensive Effectiveness
By money puck
9 years agoAlex Edler: actually pretty good at preventing quality shots
Most people in the stats community dislike plus-minus, but
my feelings are much stronger than that. I loathe plus-minus. It’s personal.
The draft for my keeper league hockey pool was about to start and I was
considering taking a flyer on a young Erik Karlsson. He was coming off his 2nd
season in the NHL, where he posted an impressive 45 points. However, at minus
30 I figured he must be in for a significant drop in playing time considering
he was obviously such a huge defensive liability to his team. Karlsson ended up
finding himself on the roster of my arch nemesis, scored an insane 78 points, and hilarity ensued. I hate
plus-minus.
my feelings are much stronger than that. I loathe plus-minus. It’s personal.
The draft for my keeper league hockey pool was about to start and I was
considering taking a flyer on a young Erik Karlsson. He was coming off his 2nd
season in the NHL, where he posted an impressive 45 points. However, at minus
30 I figured he must be in for a significant drop in playing time considering
he was obviously such a huge defensive liability to his team. Karlsson ended up
finding himself on the roster of my arch nemesis, scored an insane 78 points, and hilarity ensued. I hate
plus-minus.
In theory, plus-minus should be an immensely useful stat.
The team who scores the most goals in a game wins, so by extension it seems to
make sense that better defensive players would be on the ice for more goals for
than goals against. Unfortunately, because this statistic is so greatly
affected by goaltending and team’s on-ice shooting percentage, amongst other
factors, the usefulness of plus-minus is negligible and it has been largely
abandoned by the stats community.
The team who scores the most goals in a game wins, so by extension it seems to
make sense that better defensive players would be on the ice for more goals for
than goals against. Unfortunately, because this statistic is so greatly
affected by goaltending and team’s on-ice shooting percentage, amongst other
factors, the usefulness of plus-minus is negligible and it has been largely
abandoned by the stats community.
This got me thinking: while we know plus-minus itself is a
useless statistic, is there a way we could more accurately calculate the impact
a player has on their terms scoring differential? Or put more simply, can we
create a better plus-minus? Read past the jump to find out.
useless statistic, is there a way we could more accurately calculate the impact
a player has on their terms scoring differential? Or put more simply, can we
create a better plus-minus? Read past the jump to find out.
In my last post I discussed how the proportion of shots a goalie faces from various distances can have an impact on their save percentage, which led me to the conclude that some goalies really do have
easier jobs than others based on the quality of defense in front of them. This
led me to another question. Can we use shot distance data, together with corsi,
to isolate better or worse defenders?
easier jobs than others based on the quality of defense in front of them. This
led me to another question. Can we use shot distance data, together with corsi,
to isolate better or worse defenders?
To do that, I started to look at the proportion of shots
against various Canuck defensemen faced at even strength. In my evaluation of
goaltenders in the prior post, I included shots in all situations which the
exception of shootouts and empty nets. A shot from 15 feet out is in theory just as easy
or difficult for a goaltender to stop whether it’s a power play or even
strength, and I wanted to have the benefit of the larger sample.
against various Canuck defensemen faced at even strength. In my evaluation of
goaltenders in the prior post, I included shots in all situations which the
exception of shootouts and empty nets. A shot from 15 feet out is in theory just as easy
or difficult for a goaltender to stop whether it’s a power play or even
strength, and I wanted to have the benefit of the larger sample.
However, when evaluating the role of defensemen in their
attempt to limit shots from higher percentage areas, I felt it was important to
evaluate even strength shots only, so as to avoid skewing the data by having
defensemen who play more or less short-handed minutes. It should be more
difficult to limit shots from more favorable distances when short-handed.
attempt to limit shots from higher percentage areas, I felt it was important to
evaluate even strength shots only, so as to avoid skewing the data by having
defensemen who play more or less short-handed minutes. It should be more
difficult to limit shots from more favorable distances when short-handed.
Here is the breakdown of shooting percentages by distance
from 2013-2014 (5 on 5 only):
from 2013-2014 (5 on 5 only):
Table 1
Noting the high shooting percentages when shots are taken
inside 20 feet, you can infer that an effective defenseman is not only one that
limits shot attempts against his own team, but also one that minimizes the
amount of shots taken from these higher percentage distances. Note that I’ve
introduced the concept of the average NHL team, which is calculated as the
total shot attempts against at 5 on 5 for all teams in the league divided by 30
teams. This concept allows us to estimate many shots an average NHL team would
face at each distance and how many of those shots an average NHL goaltender
would stop. With that in mind, here are the shot against distribution of the
current Canucks defense core:
inside 20 feet, you can infer that an effective defenseman is not only one that
limits shot attempts against his own team, but also one that minimizes the
amount of shots taken from these higher percentage distances. Note that I’ve
introduced the concept of the average NHL team, which is calculated as the
total shot attempts against at 5 on 5 for all teams in the league divided by 30
teams. This concept allows us to estimate many shots an average NHL team would
face at each distance and how many of those shots an average NHL goaltender
would stop. With that in mind, here are the shot against distribution of the
current Canucks defense core:
Table 2
In theory, the better job a defensemen could do in limiting
the amount of close-range, high percentage shots, the more positive the
impact that defensemen would have on goals against. Looking at the table below,
the two defensemen with the lowest percentage of shots further away than 30
feet were Luca Sbisa and Yannick Weber, which seems to correspond well with my
opinion as to who are the weakest Canucks defensive defensemen. So far, so
good.
the amount of close-range, high percentage shots, the more positive the
impact that defensemen would have on goals against. Looking at the table below,
the two defensemen with the lowest percentage of shots further away than 30
feet were Luca Sbisa and Yannick Weber, which seems to correspond well with my
opinion as to who are the weakest Canucks defensive defensemen. So far, so
good.
My next step was attempting to see whether we could use this
data to estimate each defenseman’s contribution to goals against as compared to
an average NHL team. To do so, I took the average shots faced by an NHL team at
even strength (see table 1) then adjusted this figure for each player’s even
strength corsi to estimate the shots a defensemen’s goalie would face based on
how strong a possession player the defensemen is:
data to estimate each defenseman’s contribution to goals against as compared to
an average NHL team. To do so, I took the average shots faced by an NHL team at
even strength (see table 1) then adjusted this figure for each player’s even
strength corsi to estimate the shots a defensemen’s goalie would face based on
how strong a possession player the defensemen is:
Table
3
3
I then distributed the shots against based on each player’s
shots against distribution from each of the four distances from table 2:
shots against distribution from each of the four distances from table 2:
Table
4
4
I then applied the NHL average save percentages by distance
from Table 1 to estimate the amount of goals a team would allow after adjusted
for the defensemen’s possession (ie corsi) and shot distribution, assuming the
defensemen were to play 100% of the team minutes:
from Table 1 to estimate the amount of goals a team would allow after adjusted
for the defensemen’s possession (ie corsi) and shot distribution, assuming the
defensemen were to play 100% of the team minutes:
Table 5
Again we see that Sbisa and Weber were by far the weakest
members of the defense corps, while the remaining team members were within a
relatively tight band from each other in terms of estimated goals allowed.
members of the defense corps, while the remaining team members were within a
relatively tight band from each other in terms of estimated goals allowed.
This was getting interesting, but how do these numbers
compare to an average NHL defensemen? To figure this out, I took the average
NHL team shots against, average NHL team shot distribution, and average NHL
team save percentages by distance (table 1) to calculate the amount of goals
against an average teams defensemen would allow. I then compared this figure to
the goals against estimates for the Canucks defensemen in Table 5 to calculate
how many more goals the Canucks defensemen to prevent or allow versus average:
compare to an average NHL defensemen? To figure this out, I took the average
NHL team shots against, average NHL team shot distribution, and average NHL
team save percentages by distance (table 1) to calculate the amount of goals
against an average teams defensemen would allow. I then compared this figure to
the goals against estimates for the Canucks defensemen in Table 5 to calculate
how many more goals the Canucks defensemen to prevent or allow versus average:
Table 6
Now this is getting interesting. Assuming Ryan Stanton were
to play 100% of the even strength minutes for an average NHL team, they would
have 12 fewer goals against, or a 7% reduction in goals against at even
strength. Conversely, if Luca Sbisa were to play 100% of the even strength
minutes for an average team, they would let in 14, or 9%, more goals. Luca
Sbisa should not play 100% of the minutes for an NHL team, in case you were wondering.
to play 100% of the even strength minutes for an average NHL team, they would
have 12 fewer goals against, or a 7% reduction in goals against at even
strength. Conversely, if Luca Sbisa were to play 100% of the even strength
minutes for an average team, they would let in 14, or 9%, more goals. Luca
Sbisa should not play 100% of the minutes for an NHL team, in case you were wondering.
The final litmus test for this methodology I decided to do
was to add some well-known NHL studs and duds to the mix:
was to add some well-known NHL studs and duds to the mix:
Table 7
(Editor’s note: holy crap Jake Muzzin)
Conclusion:
So what’s wrong with this adjusted plus-minus? Well, there
are a few things. Like Corsi itself, adjusted plus-minus is impacted by quality
of teammates, quality of competition, and zone starts. Shot distance data is
further plagued by its own set of inconsistencies as purportedly shot distance
tracking can vary from one arena to the next, and of course there is the issues
with aggregating shots in the 0-10, 10-20, 20-30, and +30 foot buckets. A shot
from 2 feet out is likely to have a higher shooting percentage associated with
it than a shot from 9 feet out.
are a few things. Like Corsi itself, adjusted plus-minus is impacted by quality
of teammates, quality of competition, and zone starts. Shot distance data is
further plagued by its own set of inconsistencies as purportedly shot distance
tracking can vary from one arena to the next, and of course there is the issues
with aggregating shots in the 0-10, 10-20, 20-30, and +30 foot buckets. A shot
from 2 feet out is likely to have a higher shooting percentage associated with
it than a shot from 9 feet out.
Despite the limitations this framework has today, I believe
there are insights that can be learned from this. While we know that puck
possession is a critical factor which correlates well with team success, we
also know that not all shots have the same likelihood of ending up in the back
of the net. Finding a way to quantify both the ability of a defensemen to clear
the puck from danger in high percentage areas as well as they’re ability to
drive possession may be a start to truly being able to quantify the value of
the defensive contribution.
there are insights that can be learned from this. While we know that puck
possession is a critical factor which correlates well with team success, we
also know that not all shots have the same likelihood of ending up in the back
of the net. Finding a way to quantify both the ability of a defensemen to clear
the puck from danger in high percentage areas as well as they’re ability to
drive possession may be a start to truly being able to quantify the value of
the defensive contribution.
Limitations aside, if this framework were to help a team
find the next Ryan Stanton, then it just may be worth paying a little attention
to.
find the next Ryan Stanton, then it just may be worth paying a little attention
to.
Recent articles from money puck
