What does WAR suggest about the 2015-16 Canucks?

USATSI_8539071_168381338_lowres
Photo Credit: Sergei Belski/USA TODAY Sports

Since the dawn of hockey analytics, there has been a goal of summarizing the total value of an individual player’s contribution into a single number. The team at war-on-ice.com has made significant advancements in this area over the past year, culminating in the introduction of their Goals Against Replacement, or GAR, metric. 

GAR incorporates most of the key quantitative metrics we know to be of value including, faceoffs, penalties drawn/taken, and various shot metrics in different game states in order to calculate how many goals a player’s efforts would contribute to a team versus a replacement level player. You can read more about the background of this stat in their blog here

Advertisement - Continue Reading Below

The team at War On Ice was nice enough to let me play around with the data, in the context of what we can expect from the 2015-16 Canucks. Let’s get into that after the jump.

My first question was how closely the GAR aligns with a team’s actual goal differential, the difference between the number of goals they score and the number of goals they let in. To do this, I summed the GAR for each team to create a team GAR, and plotted that against the teams actual goal differential for the last three full seasons (2011-12, 2013-14, 2014-15). As we can see, Team GAR was actually a pretty close fit with Team Goal Differential: 

Table 1

team goal diff vs gar

Advertisement - Continue Reading Below

*Team GAR on the y axis, Team Goal Differential on the x axis

So the sum of the GAR for the various players on each team, actually aligned pretty closely with the team’s goal differential. So far so good. Next I looked at the relationship between Team GAR and Team Regular Season Points, which is key to securing a playoff spot: 

Table 2

team points vs GAR

*Team GAR on the x axis, Team Goal Differential on the y axis

As we’d expect, the alignment is slightly further apart from what we saw when comparing GAR and Goal Differential, but overall this is a pretty good fit. By comparison, here is 5v5 corsi close versus team points: 

Advertisement - Continue Reading Below

Table 3

corsi

We’ve long espoused the importance of possession for a team’s success, but as we see, Team GAR looks to have a much stronger correlation to a a team’s wins. Interesting stuff. 

Now that we’ve established that there’s a strong relationship between Team GAR and a Team’s Points, the next step is determining whether a player’s actual GAR can be predictive of their future GAR. To do this, I compared the player’s average GAR from the three preceding seasons (2011-12, 2012-13, 2013-14) to their actual 2014-15 GAR. Here are the results: 

Advertisement - Continue Reading Below

Table 4

predicted vs actual gar

As the guys at War-On-Ice.com would tell you, GAR in its current form is a solid retrospective tool, but not yet what we hope it can be in the future in terms of predictiveness. In the future, we hope to see it incorporate things like playmaking ability and defending, to add further precision. That said, the predicted GAR in Table 4 above, based on the average from preceding seasons, was actually closer to actual GAR than I had expected. 

At the very least, its a worthwhile metric to use when spit-balling on how the 2015-16 Canucks will do. In order to calculate 2015-16 Predicted GAR, I used the player’s 3 year average GAR, and calculated Predicted GAR using the equation in Table 4 above. Now to the results: 

Name 2012-13 2013-14 2014-15 3 yr Ave Predicted GAR
Forwards          
Nick Bonino 1.47 12.72 4.15 6.11 3.77
Sven Baertschi 1.67 0.00 -0.35 0.44 1.23
Linden Vey   -0.77 1.94 0.58 1.29
Daniel Sedin 10.25 17.24 4.63 10.71 5.84
Alex Burrows 7.44 0.23 10.15 5.94 3.70
Ronalds Kenins     0.33 0.33 1.18
Brandon Prust 3.77 -3.47 0.85 0.38 1.20
Bo Horvat     0.23 0.23 1.13
Radim Vrbata 2.40 7.34 9.37 6.37 3.89
Jannik Hansen 2.21 -0.29 7.51 3.14 2.44
Chris Higgins 0.07 -0.87 3.08 0.76 1.37
Henrik Sedin 11.94 18.64 11.93 14.17 7.39
Derek Dorsett -2.80 3.37 -4.72 -1.38 0.41
Defense          
Matt Bartkowski 0.20 -4.20 -4.49 -2.83 -0.24
Dan Hamhuis 6.73 5.89 -2.31 3.44 2.57
Alexander Edler 4.00 6.13 6.20 5.44 3.47
Frank Corrado -0.52 0.64 0.52 0.21 1.13
Christopher Tanev 4.63 8.37 9.73 7.58 4.43
Luca Sbisa -4.46 -2.48 -9.44 -5.46 -1.42
Yannick Weber 1.31 -1.42 1.93 0.61 1.30
Adam Clendening     -0.14 -0.14 0.97
Goalies          
Ryan Miller 11.00 13.99 -3.12 7.29 4.30
Jacob Markstrom -9.62     -9.62 -3.29
        54.30 48.10

*Note: GAR data in table above varies from that currently available on waronice.com as the website has not yet been updated for end of season data. 

Using this method to calculate Predicted GAR for the 2015-16 Canucks roster, as currently constructed, we estimate that the Canucks would have a Team GAR of 48.1, which would roughly equate to a 92 point season. Using the 3 year average GAR of 54.3, we would expect the Canucks to have a 93 points season. Using the data information that GAR gives us, we would expect that the Canucks will be on the outside of the playoff bubble next season. 

Conclusion

During the course of the year, there has been a number of interesting developments in the analytics community, and GAR is definitely a stat to keep in mind when evaluating a team’s performance, and prospects for future success. By this measure, it would appear that we should expect the Canucks to slip outside of the playoff picture next year. What do you think? 



  • Dr. StrangeTiger

    So what I generally like about the use of metrics is the ability to provide insight into performance and value in a way that we don’t get from the “eye-test” or something similarly gut-based. I’m not entirely sure of the value of this method though, especially as you have yourself acknowledged that this is a tool for assessing past performance rather than a predictor for the future. I don’t think it takes GAR to tell us that the Canucks have an outside shot at the playoffs given their aging first-line players, lack of prospects with enough experience, and questionable choices in supporting cast.

    Moreover, it’s hard to necessarily see what value this adds to the contextual evidence I’d already have on hand to know why Bonino’s numbers fluctuate the last three seasons, with the outlier being the 13-14 season skating with Getzlaf or Perry, or that Burrows rebounds nicely from the injuries and Tortorella effect. It would be more useful in explaining this tool why it shows Daniel as head and shoulders above the rest of the F while Henrik barely squeaks ahead of Vey and is eclipsed by Hansen. Or why we should pay attention at all to this in the case of Kenins, Markstrom, Clendenning, Horvat, Baertschi, those with not enough of a sample size due to a lack of opportunity, misuse or age.

    I don’t know, I just wonder at the value of this tool. I love anything that would finally convince teams that the Prusts, Dorsetts and Sbisas aren’t worth investing in but I’m not sure this is it.

    And the suggestion that the Canucks will miss the playoffs is neither a newsflash nor needs this metric to predict it.

  • SISMIM

    So what I generally like about the use of metrics is the ability to provide insight into performance and value in a way that we don’t get from the “eye-test” or something similarly gut-based. I’m not entirely sure of the value of this method though, especially as you have yourself acknowledged that this is a tool for assessing past performance rather than a predictor for the future. I don’t think it takes GAR to tell us that the Canucks have an outside shot at the playoffs given their aging first-line players, lack of prospects with enough experience, and questionable choices in supporting cast.

    Moreover, it’s hard to necessarily see what value this adds to the contextual evidence I’d already have on hand to know why Bonino’s numbers fluctuate the last three seasons, with the outlier being the 13-14 season skating with Getzlaf or Perry, or that Burrows rebounds nicely from the injuries and Tortorella effect. It would be more useful in explaining this tool why it shows Daniel as head and shoulders above the rest of the F while Henrik barely squeaks ahead of Vey and is eclipsed by Hansen. Or why we should pay attention at all to this in the case of Kenins, Markstrom, Clendenning, Horvat, Baertschi, those with not enough of a sample size due to a lack of opportunity, misuse or age.

    I don’t know, I just wonder at the value of this tool. I love anything that would finally convince teams that the Prusts, Dorsetts and Sbisas aren’t worth investing in but I’m not sure this is it.

    And the suggestion that the Canucks will miss the playoffs is neither a newsflash nor needs this metric to predict it.

    • SISMIM

      Never being in the playoff picture next year would be even better. My big fear for next year is that we are still in the hunt at the end of January. We need to accept that we need to rebuild, but management is content to kick that can down the road.

      Also, GAR says Sbisa is more valuable than only Prust, Dorsett and Bartkowski. Worth noting that Kassian has a higher GAR than even Henrik.

      JB will be hoping there is no merit to it.

  • Dr. StrangeTiger

    further to PB’s questioning of the value of this tool I would add that “predictive” tools are used on past performance, assumptions and generalizations.

    I wonder that had this tool been based solely on 2013 where Torts tanked the team it would likely have predicted, on past performance, that the Nux would finish with about 90 pts in 2014……instead they managed 101 points and a playoff spot.

    There is no increment for intangibles such as individual pride, a player having a career year, the change in team morale based on an infusion of youth and new players, coaches and leadership etc. etc.

    So, using a tool that only measures past performance seems a bit wonky.

    Even Financial Institutions warn–in writing–that “past performance is no guarantee of future performance” and financial predictivity is a much more lineal process than trying to predict the outcome of a group of 20+ individuals and their past performance….many of which were not on this team, in this league in the past.

    it also cannot predict injuries, slumps, trades etc.

    So, while I actually agree with the summary comment that the 2015 team is likely going to be “playoff-challenged” (and as per PB above), I think this exercise is open to question as it measures past performance as a predictor of future performance(s) as its sole focus.

  • SISMIM

    I like the approach and the results are very interesting. In terms of predicting next year’s GAR, did you try using extrapolation methods (using say the last 3-5 seasons) instead of taking an average of previous seasons?

    • SISMIM

      Thanks. I looked at a few methods actually, including changes in GAR as players age. Interesting, the correlation between prior and current year GAR gets stronger as players age. Intuitively, this seems makes sense.

      Definitely lots of opportunity for future work, but the purpose of this post was more to provide an high level intro to GAR.

      • Dr. StrangeTiger

        You know there is an easy way to test your GAR model’s ability to predict future team points right? Use the first 2 years data to build the model, then use said model to predict the third year of data. Plot predicted vs actual on a 1:1 line and calculate goodness of fit stats like a pearson’s r, RMSE, R2, etc. If you want to do it right, use a linear mixed effects model and treat team as a random effect to account for the repeated measures, although team is only partially repeated considering changes in personnel between seasons.

  • Mantastic

    GAR is such a dumb stat to use on a whole team. You’re looking at this as if no other team will either improve or get worse, thus either depressing/inflating the points earn in the season.

    WAR is great for 1 player’s importance in a vacuum in baseball. But is never great at determining how a whole team will do.

    the R^2 values shows that there is a large variance within this model, even if it is relatively linear, it’s really not a good fit. with a name like moneypuck, i would assume you would be much better with numbers but sadly, you’re not

  • Dirty30

    I would like to know how the GAR Of other teams correlates with the Canucks GAR — obviously teams they seldom play have less effect than teams they play often. But if the predicted GAR for next season is s predicated on teams remaining static and only the Canucks changing then I don’t think it has much utility at all.

    Just remember, all the hoopla about ‘change’ really resulted to n the Canucks winning five more games last season than during the Torts Terror.

  • Mantastic

    Every one is already putting in their various complaints/suggestions so I might as well add one more. I know things are hugely statistically limited but for things like this it always feels to me like it would be more meaningful if you’d give some more information about this final point estimate by looking at the spread in points in table 2 for teams with a certain combined GAR near the value you gave for the Canucks (like +/- 2?) It seems to stretch down as low as 75 points and as high as 105, with the bulk being contained inside of ~[82,95] (like +/- 1 sigma, or a full width half max type thing). It just feels like adding this type of additional information would do a better job of at least attempting to qualify the final statement of this little study while also adding a little more weight to the it as well?

    • Dr. StrangeTiger

      Not really. Money puck already stated that GAR doesn’t take playmaking into account when calculating GAR. I suspect that, given this deficiency, and that roughly 35% of next year’s team having a sample size issue, that the model probably has a very large margin of error. On a pure guess level, I’d say +/- 10% or so (ie: the margin of a ‘bubble’ team making or missing the playoffs).

    • SISMIM

      I noticed that too. And while the article notes that “GAR data in table above varies from that currently available on waronice.com as the website has not yet been updated for end of season data,” I’m a little suspicious as to why Henrik’s season numbers are so different (when comparing those posted in the article versus those posted on war-on-ice), while Daniel’s (and all of the other players that I looked at) are fairly similar (between the website and the article).

      Here are the “total GAR” numbers for Daniel Sedin, as currently posted on war-on-ice:

      2012-13: 10.78

      2013-14: 16.88

      2014-15: 2.86

      And here are the article’s numbers (for Daniel):

      2012-13: 10.16

      2013-14: 14.14

      2014-15: 7.47

      Seems reasonable. The 2014-15 numbers show the largest difference, which makes sense, given that “the website has not yet been updated for end of season data.”

      But then you look at the “total GAR” for Henrik Sedin, as currently posted on war-on-ice:

      2012-13: 12.13

      2013-14: 17.94

      2014-15: 12.01

      Those are actually better numbers than Daniel’s (which is more in line with what I’d expect to see when comparing the twins)

      And, once again, here are the numbers from the article (for Henrik):

      2012-13: 1.75

      2013-14: 1.71

      2014-15: 2.74

      Much lower GAR than Daniel. Surprisingly so.

      I’d be very interested to learn the reason why Henrik’s GAR numbers in the article are so different from those currently posted on war-on-ice? I understand the website’s numbers aren’t “current” and that the “end of season data” will change things, but I’m not sure how that accounts for the huge differences between the numbers for 2012-13 (12.13 versus 1.75) and 2013-14 (17.94 versus 1.71)?

      Something looks “off” to my eyes.

      Based on the numbers currently available on war-on-ice, I’d actually expect Henrik to have a slightly higher GAR than Daniel for each of the seasons listed. This would result in a higher average GAR and predicted GAR than what is currently shown for Henrik (which would ultimately raise the total team GAR and alter the projections in the article’s conclusion).

  • Mantastic

    I don’t have a problem with metrics if they are used to indicate a possible trend in a player or team. This makes them a valuable tool in the management of that player or team. I do have a problem when they are used to predict the future which WAR appears to try and do. There are so many variables such as unsustainable shooting percentages, slumps because of injury, change in teams, aging, etc. etc. which haven’t been taken into consideration and will skew the results. I know some people would like to see the GM (and possibly the coach) replaced by a computer but, hopefully that is a long way off.

    • Mantastic

      As noted in the article:

      “As the guys at War-On-Ice.com would tell you, GAR in its current form is a solid retrospective tool, but not yet what we hope it can be in the future in terms of predictiveness. In the future, we hope to see it incorporate things like playmaking ability and defending, to add further precision.”

      I don’t think money puck is making a claim that it can predict the future, as you said. He was speculating it’s potential value in predicting future performance, which is fine, IMO.

  • Mantastic

    I’m not good enough (or interested enough) at math to do any of this number crunching myself, although I love all these new evaluation tools for players and teams. That’s why I hang out here.

    With that said, any stat that says Jannik Hansen is a better hockey player than Henrik Sedin may need some fine tuning.

  • Mantastic

    You use 3 previous years GAR to predict 2014-2015 GAR. Unless I missing something crucial, the slope should be 1 here (the spread should occur above and below the 1:1 line) since some players will exceed their average GAR with a good year in 2014, and some will fall below their average GAR in 2014 with a bad year. There seems to be something seriously wrong with your model if you are predicting the GAR of a player using previous GAR and the slope turns out to be .44. Why such a reduction in GAR for 2014 vs the previous 3 years? I suspect the issue lies with applying a team level metric to individual players, but with the level of description of the stats here its pretty much impossible to tell. Also, you have repeated measures in time in your data (player) that are not accounted for in simple linear regression.

    Beyond that, this metric seems to be pretty much useless if it predicts Hansen, Hamuis, Edler, Tanev, Vrbata, Burrows, are all more valuable players than Henrik.

    Again, good effort, but the math in your model doesn’t hold up, and the results seem nonsensical to anyone who has actually watched the canucks.

    • Mantastic

      Good to see you back, stranger.

      .. Yes, there may be issues with applying an individual stat like GAR to the team level. More work needs to be done here. Plus some of the individual results (H. Sedin/Sbisa specifically) are head-scratchers, which are likely indicative of refinement needed in the GAR model.

      That said, I think an interesting metric which I hope led to an enjoyable read.. Or not..

  • Mantastic

    I clicked on the link you provided (http://blog.war-on-ice.com/the-road-to-war-series-index/) to read more about GAR but didn’t see an article about it at that link (unless I’m blind). Is there a specific article to read to learn more about it? You kind of skim over what GAR is so it would be nice to get a clearer sense of what this stat entails.

    On another note, thanks for introducing this stat. I struggle with math/stats quite a bit (terrible at math/numbers) but I do my best to try to grasp the concepts and it’s nice to know whats out there.

  • Dr. StrangeTiger

    GWAR!

    So Gillis is now going to be a University professor in Victoria…., he was very up to date on these numbers, but can’t seem to get another gig in the NHL. Quinn, Burke Torts all got second, third Ect..,,

  • Dr. StrangeTiger

    “What does WAR suggest about the 2015-16 Canucks?”

    If there’s one thing I like about monsieur MoneyPuck it’s his titles.

    It’s like an open invitation to insert my jokes about the Canucks.

    And for that Monsieur moneyPuck…. merci.

  • Dr. StrangeTiger

    When comparing GAR to points, are you taking out extra points awarded in the shootout?

    I didn’t catch that note in the article, but it might make the comparison more relevant.

  • Dr. StrangeTiger

    Ahhhh the sedan twins! The demise of the Canucks! I love it! Instant rebuild when they fall….. Again… Vancouver is getting nothing in return for those bros

  • SISMIM

    Ah, statistics, ya gotta love ’em. I recall a hockey hall of fame defenceman saying in a loud voice “statistics are for losers.” Also coming to mind is the opinion of an old high school English teacher who came up with the phrase “trash, balderdash and piffle” when describing the ridiculous theories of a student he didn’t particularly care for. This guy never saw a hockey game in his life, yet his musings seem appropriate in today’s world.