On the Proper Use of "Real-Time Stats" or, Why We Don't Use Blocked Shots to Judge Player Performance

Cam Charron
March 20 2012 10:58AM


Keith Ballard, shown here putting his body on the line for the team, is easily the Canucks best shot blocker.
But would you describe him as the team's best defensive player?

The objective, numbers centric approach we pride ourselves on at Canucks Army isn't intuitive and admittedly some of the concepts can be challenging. As such, I figured it might be informative if I explained in depth why I've taken umbrage at the way several analysts have misused the numbers over the past week.

Read past the jump for more!

During the Vancouver/Columbus game, Province Sports beat-writer Ben Kuzma had this to say about the Canucks' performance:

Obviously, showing "jam" contributes to winning hockey games. If one team is so far ahead of the other in "jam," the numbers on the scoreboard barely matter! 

Mostly, "heart" "grit" "jam" "toughness" or whatnot are buzzwords used by people as intangible qualities to prop up their arguments or observations. The new "real time" statistics tracked by NHL.com in recent years have allowed people to make these intangibles more tangible, but too often these "real-time statistics" just aren't used in the proper context. 

I once saw a Chicago fan last season explaining that Detroit isn't a good defensive team by pointing to the club's low shot-block totals. Fact is that Detroit has low shot-block totals for two reasons:

  1. Their in-arena scorer is very stingy in awarding both missed shots and blocked shots, and
  2. They always have the puck in the other team's zone, which means they there aren't a lot of opportunities for Detroit to block shots in the first place.

Now I don't mind it when people use statistics to back up their argument, that's what I do for a living! But I take issue with the fact that a lot of people (this goes for general fans as well as mainstream analysts) use weak statistics to back up their argument when there are better alternatives at stake.

This doesn't mean that shot blocks don't mean anything. Oh no, they mean lots. In fact, I calculated a pretty strong correlation over the last three seasons between the number of shot blocks and the number of points per game a team gets. The problem is that the teams that block the most shots are the ones who have the fewest wins:

Now that's not necessarily a huge correlation, but the r-squared is .144 (0 implies no correlation, 1 implies perfect correlation) and you'll notice the trendline is negative. Of course, this doesn't necessarily imply that blocking shots causes losses (and blocking a shot is certainly preferable to the alternative of letting the shot through) but it does indicate that a huge deficit in total shot block numbers isn't necessarily a positive.

There's a reason why we use the numbers we do at Canucks Army. We pick things that are both repeatable, predictive, and correlate with winning. Scoring chances, Fenwick, Corsi, these things are all advanced stats, advanced implying they're better than what you can get in the NHL.com events summary (although we link to it every night because there are always good nuggets in there. The event summary splits up even strength and powerplay shots, it shows you the number of shot attempts per team and player, and also includes faceoff data - all essential information).

One of the first things Bill James attempted to do in his 1977 Baseball Abstract was re-think how people judged defensive play. Going back to my earlier point about the Chicago fan, good defence shouldn't be qualified by the number of blocked shots a team has, but rather how many goals it gives up. In the end, that's what counts. Good offence can't be qualified by speed, forechecking, skill, vision or any of those intangible qualities, but whether or not the puck goes in the net more than a replacement.

After the Vancouver/Minnesota game, ex-Canuck turned analyst Garry Valk had this bit of analysis:

Obviously, I never played at a high level. But neither did a lot of good General Managers, who became lawyers or agents at the end of their playing careers after taking some form of higher education. While Valk did play at a higher level, I'm not sure that earns him a free pass to use hits (as if they indicate anything worthwhile) in his analysis. Maybe in his day, coaches paid attention to whom was hitting whom, but I'd trust the Canucks brass to stay away from something so inefficient as "who had the most hits?"

I replied with this, and the proceeding image will show you why:

Hits to points percentage over the last three seasons has had an r-squared value of .0002, so it really doesn't matter. Part of the problem is the way that the NHL records hits is messed up, because different arenas have different criteria. You could judge "hits for/hits against" as a rate stat if you really wanted, but I don't think you'd get really far if you did. JP Nikota has looked at "zone hit ratio" as determining where the hits took place (ie, hits the defensive zone aren't as good as hits in the offensive zone) but he couldn't really find a reason to think it synced up with puck possession.

Certainly, that doesn't mean "don't hit," but it means that hits may not be the best way to look at whether or not the Canucks' bottom six (who were awful by our scoring chance data) were effective or not. It's also not really an accurate measurement of whether or not a team is "hard to play against," and it's worth noting that last years Boston Bruins, undoubtedly the NHL's version of the Baltimore Ravens, were 22nd in the league in hits last season. Physical play matters, but it doesn't necessarily show up in the "hits" statistic. 

I'm sure you could find several games where the Canucks bottom six out-hit the opposition by a good margin and got crunched in possession. More importantly, hits just aren't the objective stat that Valk seems to think they are: they're entirely subjective, and based on the bias of each arena scorekeeper. It's much like the "error" in baseball that Bill James attempted to remove.

We like to use scoring chances as the best mix between "puck possession" and "shot quality" statistics. When we don't have available scoring chance data, we use Fenwick, because it is an underlying number that correlates highly with scoring chance data: players that have a high Fenwick—a shot differential statistic that includes missed shots—number will also have a high scoring chance +/-. We know this is true. And we know that Fenwick is important:

Like Vic Ferrari wrote in the above link, "If you can't spot the pattern there, well you're just not trying".

There is a reason that Canucks Army uses the numbers we do, and it's not because we're trying to thumb our nose at mainstream media folks (though we do that sometimes too). It's because these are the superior numbers - they're shown to be predictive of future wins and losses and we're just doing our best to give our audience the most useful information available.

63811cbf517d2d685ea09e103488ea3a
Cam Charron is a BC hockey fan that writes about hockey on many different websites including this one.
Avatar
#1 intheloo
March 20 2012, 11:37AM
Trash it!
0
trashes
+1
0
props

Just a quick thought, has there been any analysis done on the correlation between Blocked Shots as a % of Corsi Events vs. Points %?

Avatar
#3 MathMan
March 20 2012, 11:50AM
Trash it!
0
trashes
+1
0
props
Cam Charron wrote:

I'd check, but we don't have shot block data for each team as a rate at EV, which is what this would be good for.

If you have Corsi, and you have Fenwick, getting blocked shot rate would seem to be a simple matter of subtraction, no?

Otherwise, fantastic article on a point that bears repeating until it settles in. Blocked shots, individually, are not bad plays, but it should be a red flag if a team or player is forced to block a lot of shots. Another similar point I like to point to, quite beyond the massive area bias, is giveaways -- individual giveaways are bad plays, but having lots of giveaways means that you a-have the puck a lot and b-are trying to pass it which may tend to indicate you are a strong possession player. Just like blocked shots and hits -- the real-time stats do not necessarily have the significance one might ascribe to them intuitively.

Avatar
#4 MattM
March 20 2012, 11:50AM
Trash it!
0
trashes
+1
0
props

Hey Cam, do you know if anyone has ever done any analysis on a correlation between winning and (shot blocks/corsi -)? Obviously, the big problem that shot blocks suffers from is that it's easier to aggregate a large total when you never have the puck. If we convert it to a ratio of blocks per shots attempted against, would we get some more signal out of the noise?

On a larger point, do we see any repeatable talent on an individual or team basis on any metrics along this line? People often talk about forcing teams to take lower-quality shots, which at its logical conclusion is taking shots that are more likely to be blocked or missed as well. Can we try to establish via a ratio of (shots - / corsi -) if a team is legitimately allowing more, lower quality shots?

Avatar
#6 Jay
March 20 2012, 12:10PM
Trash it!
0
trashes
+1
0
props

I really like reading these types of articles here. I've really developed an interest in advanced hockey sabermetrics over the last few months. Where can I learn more about it?

As I am starting my masters in the fall i am currently looking into ways I can possibly tie a CS masters thesis with hockey statistics.

Avatar
#7 Jay
March 20 2012, 12:52PM
Trash it!
0
trashes
+1
0
props
Cam Charron wrote:

We don't have team-level Corsi yet, unfortunately, just player-level, and team-level Fenwick.

I could go through some 90 timeonice scripts, but that would be excessively time consuming... although it would provide us with some good information, which I guess is our mission statement.

I'll see what I can do.

(I don't know how to only quote you, not to quote both).

Rather than going through the timeonice scripts one at a time couldn't you program some script to scrap all the data and dump it into something like a csv file that excel can read?

Avatar
#9 Kent Wilson
March 20 2012, 03:03PM
Trash it!
0
trashes
+1
0
props
Jay wrote:

I really like reading these types of articles here. I've really developed an interest in advanced hockey sabermetrics over the last few months. Where can I learn more about it?

As I am starting my masters in the fall i am currently looking into ways I can possibly tie a CS masters thesis with hockey statistics.

Hmm. I might be able to help you with that.

Contact me at kent.wilson@gmail.com if you're interested.

Avatar
#10 Jay
March 20 2012, 03:31PM
Trash it!
0
trashes
+1
0
props

@Cam Charron

I am one - if you need any help with that kind of stuff let me know, twitter @joshweissbock or josh.weissbock@gmail.com

Avatar
#11 BSH Eric T.
March 21 2012, 07:51AM
Trash it!
0
trashes
+1
0
props

@Cam Charron

Cam,

You could also grab BtN's Excel file with the player data and calculate it from there.

Make a column where for each player, you calculate the total Corsi while that player was on the ice (Corsi/60 * GP * TOI/60 / 60) and shot blocks while that player was on the ice (BA/60 * GP * TOI/60 / 60).

Then you can use Excel's sumif function to get the total Corsi and blocked shots for each team, or you can sort the list by team and sum them by hand. It'll be a little bit off because of players who switched teams, but close enough.

Avatar
#12 BSH Eric T.
March 21 2012, 09:14AM
Trash it!
0
trashes
+1
0
props

Erm, realized after I posted that that it's a little more complicated than that, since Corsi is the shot differential and you just want the offense part.

Same idea though, just sum GFON/60, SFON/60, MFON/60, and compare to BAON/60.

Comments are closed for this article.