53

On bias-confirming video and the definition of objective analysis

Every video analysis article ever has received the same comment: “I could collect clips of ten bad plays [insert team’s best player] made and argue he’s the worst player on the team.”

I’m sure you could – but that’s not how analysis works.

It is almost impossible for humans to turn off their biases and look at something old like they’d never seen it before. As a result, it is extremely difficult to take an unbiased and objective look at a player. But what does “objective” even mean?

From the Cambridge Dictionary:

Objective – based on real facts and not influenced by personal beliefs or feelings.

So, to get back to the opening quote, if you just take ten bad plays and argue the involved player is bad at hockey, that’s neither objective nor valuable. Yet, writers – including myself – write entirely one-sided pieces again and again, and we have the guts to call the result “objective analysis.”

And there’s a reason for that.

Let’s assume a team’s defenceman sticks out with bad possession numbers. Upon further research, his failure to exit the defensive zone with control seems to be a significant reason for it, although he’s known as a strong skater and puck-mover. Does that mean the numbers are lying and stats are stupid or are they accurate, and if so, can these assertions be supported with video?

This, precisely, is what video coaches exist for, and it is what writers like myself aim to provide the public. So, we dive into video of maybe five or ten games from the past few weeks and look at every breakout said defenceman has made.

Maybe we find the player spent most of his ice time with replacement-level players who don’t get open properly. Perhaps he is frequently matched up against opposing teams’ best forechecking lines, who are too good at shutting down the offensive blue line and the neutral zone. Or maybe the video reveals a lack of awareness, vision, or passing skill.

Whatever our findings may be, we now want to present them to you. Or, in a video coach’s case, to the head coach and the player.

And this is where things get tricky. We don’t want to let our biases rule our thoughts and resulting work, and come up with an entirely negative video package, only because it’s what we wanted to find. We don’t want to include plays we might not have picked up as negative had they been made by a different player but use them anyway because they make this player look bad.

But at the same time, there is absolutely no point in showing you the one time the defender played a perfect stretch pass for a breakaway or the time he fooled a forechecker with a quick fake when he still only breaks out of the D-zone successfully 40 percent of the time.

We shouldn’t go into video analysis thinking “I’m going to make this guy look bad at hockey.” But there is absolutely nothing wrong with looking at statistics and thinking “I am going to find the reasons for this player’s numbers.” In fact, NHL clubs hire people to do precisely this.

Being objective does not equal showing some good and some bad clips of a player. Being objective means going into video analysis open to being proven wrong. But if we are proven right, there is nothing wrong with wrapping that up in a one-sided piece that focuses on nothing but examples of bad plays.

The goal is to dive into video and look out for trends, while shutting off biases as much as humanly possible. What matters isn’t the one time a player did something surprising. What matters are all the times he did the same thing, and what results from those plays.

If our struggling defenceman shows examples of excellent breakouts and clearly displays the skill-set to provide that on a consistent basis, the goal would still be to find out why it hasn’t been working for him. And in a next step, the coaches and the player himself could use the wrong examples to focus on using his skill-set more efficiently and successfully.

So, the next time you get angry about that negative article about a player you like, maybe consider that even an entirely negative thing can be objective analysis.

  • Locust

    It has nothing to do with “a player you like”.

    Most here don’t agree with the hatchet job that Canucks Army and specifically JD consistently gives to Sutter, Guddy and Dorsett. All three have had stinker games – and that has been pointed out – and agreed to. But when they play awesome, CA either ignores it or tries to change the narrative and move the goalposts. JD leaving out any “stats” about their last great game is direct proof of that.

    When will anyone here just admit that fancy stats don’t tell the whole story and there are a dozen things that will never show up in fancy stats that make players important and worth every penny they are paid. Simple. Then you can move on from these immature posts about us “just not getting it”.

    Boy, you guys just don’t get who “butters the bread” here do ya…….

    Trying to blame us for you not being able to at least ‘agree to disagree’ is laughable.

    • Cageyvet

      I agree whole-heartedly and would add that including good plays would serve a few purposes. It would alleviate much of the perceived bias, it could be a huge component of evaluating a still-progressing player such as Pouliot, and would also help veteran players try and repeat those “surprising” good plays.

      The premise here isn’t unfounded, I understand the point behind backing up the stats with video. That’s completely valid. It comes across as a reverse highlight montage, however, we all know there is more to the story whether it’s a positive or negative-bias collection of clips.

    • Deliverator

      I agree with the guys at CA. Sutter, Guddy and Dorsett are heavily overrated and over used. I don’t mind Sutter and Dorsett in a 4th line role, but they are horribly overmatched as a shutdown line playing against the other teams’ top lines.

      • Dirk22

        Careful Deliverator. You’ll need to come up with some new kind evidence to back up those claims. Numbers and video isn’t enough for most people on here anymore.

  • “So, the next time you get angry about that negative article about a player you like, maybe consider that even an entirely negative thing can be objective analysis.”

    Yeah, but just because something is objective doesn’t mean it can’t be wrong:

    We Were Wrong About Bo Horvat – Jackson McDonald
    I Was Wrong, And How: Adam Gaudette – JD Burke
    I Was Wrong, And How: Matthew Tkachuk – JD Burke

  • Laxbruh15

    You’ve once again managed to completely miss the point. You creating articles that specifically collect only negative plays when the positive plays out number the negatives by a margin of five to one in reality is the issue. You’ve repeatedly attacked gudbranson despite the fact that he’s been awesome this season. He has a goal differential of +58.33 despite being played against the best players on the other team almost the entire time he’s on the ice, it’s also +6.06% above the team average. His numbers are phenomenal and elite at this point in the season. Yet despite this, your coverage would indicate the exact opposite. His good games or even games where he isn’t bad, by your standards, are ignored completely. Instead you isolate individual plays and try to blame him for them. This is just an extremely wordy article that doesn’t contain much of a message to attempt to excuse deceitful reporting.

        • crofton

          Railing against analytics? LOL ROFLMAO Those are numbers I haven’t seen, and I have never railed against analytics. In fact the opposite if those numbers are accurate. Have you seen those numbers? I think I can safely say they have never made an appearance in any JD, Cat or Pet Bugs article. Is it at all possible for you to post without an insult? Why not show he’s wrong if that’s what you think?

    • Freud

      I’m having deja vu after reading your post. Somebody once said, if you say it over and over again doesn’t make it accurate.

      You didn’t even go for expected goals for? You’re picking goals for and against – that’s the same as plus/minus. Deceitful or super ignorant? I’m going super ignorant.

      I’m here in my mommy’s basement looking at the numbers. How do you explain Gudbransen’s scoring chance differential? 99 for and 150 against for 39%. The worst amongst all defence man on the team. Of the 160 d-men who have played 200+ minutes this season, that ranks Gudbransen 157th.

      • Roy

        Yes, but he hits, so the statistics are meaningless. The comments section only use the Grit-O-Meter (TM), which means every time they roar with toxically masculine approval when Gudbranson smashes someone (behind the play, out of position, etc notwithstanding), the amount of “eye-test” posts they will each make in his favour/defence go up by three each. Just drive-by post the word “grit” and you’ll get 76 likes and a lot of figuratively wet, monosyllabic grunting by way of commentary.

    • Deliverator

      These are a sample of Gudbranson’s analytics from Hockey Reference:
      CF: 42.9, CF Rel -8.9, FF 41.5 FF Rel -8.9, PDO 103.2, oZS 40.9

      Not great, except for the PDO. I can’t find a single reference for your GD, and since you haven’t given one, the safe assumption is you pulled it out of your ass.

  • Gampbler

    The objectivity argument is out the window as soon as we talk about successful d-zone exits. What is a successful d-zone exit and who decides what constitutes that success? Is an uncontrolled chip off of the glass not the better hockey play in some instances? Who do we assign blame for an unsuccessful pass, the passer or the receiver? Do we blame a defensive partner for putting their counterpart under all sorts of pressure or do we merely assign blame to the defender that directly fails at leaving their own zone? As much as we’d like to think this is science, it never will be..

    • crofton

      I have asked myself all those same questions, and I don’t think stats can provide satisfactory answers. If you had 10 advanced stats people collecting stats on each game, and then had them switch games, so each of the 10 did all ten games, I can pretty much guarantee that the reported numbers would be different for all 10 games. Kinda like having 100 referees decide what was goalie interference, too many interpretations. And the stats guys are only human, so I believe they allow their personal biases to show. The worst part of this site is the seemingly uber reliance on Corsi alone. Well that and their personal biases

    • Holly Wood

      All coaches love and preach the “chip off the glass” instead of trying to force a play that’s not there . The guys that don’t appreciate that play have never played above peewee house.

    • Janik Beichler

      Sometimes, there is too much pressure to make a controlled play. A controlled play is generally better, though. You don’t want to give the puck away as soon as you get it. So when a player consistently fails to get the puck out of the zone while keeping his team in possession, can we not agree that’s bad?

      As to assigning blame and variance between different stats trackers, this was addresses in the article. If the numbers show something that stands out, we have to go in and try to see why the numbers are what they are. If we find the defender makes the right plays consistently but others mess up, that’s a valid conclusion. If we find the stats tracker seemed to assign blame oddly, that’s a valid conclusion.

      “This, precisely, is what video coaches exist for, and it is what writers like myself aim to provide the public. So, we dive into video of maybe five or ten games from the past few weeks and look at every breakout said defenceman has made.

      Maybe we find the player spent most of his ice time with replacement-level players who don’t get open properly. Perhaps he is frequently matched up against opposing teams’ best forechecking lines, who are too good at shutting down the offensive blue line and the neutral zone. Or maybe the video reveals a lack of awareness, vision, or passing skill.”

  • McGretzky

    The authors on this blog and others have a logical goal that usurps “objective” analysis: career advancement.

    Considering the poverty line is a notch above hockey blogger, there is zero reason to believe the public will get “objective” analysis anytime soon.

    If/when the public gets “objective” analysis from hockey bloggers, it will be a decade or two down the road when the NHL is more like MLB and Ivy league analysts get turned down.

    Hence, the Beichlers and Burkes of the blogosphere are not talented, educated or financially independent enough to provide “objective” analysis.

    • Roy

      You’re saying bloggers on this site are not worth believing because you think they’re poor? That’s pond scum level of correlation. Lower than pond scum, in fact.

    • jaybird43

      Come on. Janik has reasonably stated a reasonable case and discussed various cognitive biases that effect high level thinking. He says, essentially, look at some stats, then dive in deeper to look at actual video.

      Who was the player up against, how was he deployed. Stats don’t totally tell you that (yet). So what’s the problem with what he’s saying? It makes sense to me.

  • goalfiSh

    This site is less and less about the Canucks each day, and more and more about this imaginary battle of “advanced” stats vs the world.

    I remember the days when Thomas Drance used these stats to enlighten his already insightful, well rounded and *actual* objective analysis of the team.

    Now it’s a about the personal feelings of this crew of butt hurt writers who have taken one to many poundings in the comments section to actually be able to produce anything remotely objective.

    Simpky put, this site sucks.

  • Rodeobill

    I agree with this article, with the caveat that said opinion is framed in an objective way. I would rather see something like “Gudbranson has trouble clearing the zone” as opposed to “Gudbranson sucks eggs, and here’s the numbers to prove it!”

    Looking at curious numbers, framing a sensible hypothesis to explain them and providing evidence to support the argument is great. This is an excellent place to begin a good argument, the present idea could find further support in discussion or alternative evidence and hypotheses could be proposed. As long as the article tries to keep objective (or even in a positive light to appease fans/readers of the article, i.e “Gudbranson can could really improve his game by addressing X,Y,Z”) I am good with that. I even appreciate it. These kind of articles give me insight into game and team I previously did not consider. Objectively framed opinions (not attack pieces) leave it up to the fans to reconsider their opinions without feeling defensive after, as said in the article, the writer is specifically attacking a cherished favorite of the fan reading.

  • GLM

    “So, the next time you get angry about that negative article about a player you like, maybe consider that even an entirely negative thing can be objective analysis.” That’s the right attitude, write an unpopular negative article about beating a dead horse and your only takeaway is that your readership is biased and wrong. You’ll fit in well with the rest of the writing staff I can tell.

    • Whackanuck

      …but he’s right about certain circumstances. Saying, for example, that Canucks Kassian was consistently out of position and proving it with numerous (the great majority) of clips can be objective. If a fans eye test says he’s a good player and his analytical numbers consistently show he’s not, there’s a bust. The answer IS that he blew his defensive coverage and he needed to change. That’s all negative and still objective. I think where things have gotten off track is the few snide, condescending writers are off-putting not so much for their points but how they say it. The other is the insistence that past stats can reliably predict a future outcome. Returning to the Kassian example, clearly, his partying ways affecting his play couldn’t be elicited from his “analytics”. Nor could his relative life improvement be predicted by analytical data. As for the commenters, the eye test suggests they don’t enjoy being patronized but htere aren’t any measurable stats that signify they do.

      • GLM

        Right, I wasn’t really taking issue with the legitimacy of the writer’s argument or him only using negative clips to support his advanced analytics. The writer could very well be correct that Gudbranson is flaming hot trash because the advanced stats say so, I wouldn’t know. I do know though, that the article written was controversial and not so popular.

        So I feel you hit the nail on the head saying that “the few snide, condescending writers are off-putting not so much for their points but how they say it”, because I feel all too often the writers at CA are maybe too concerned about proving their ideology right and maybe self-serving their own interests than they are writing an interesting original piece that would appeal to a Canucks fan checking in on CanucksArmy for the day.

        Because I don’t think anybody can deny that the article written was a negative topic that’s been run into the ground. So, for the writer to think the only reason his piece was poorly received was simply because this readership base “is biased and irrational”, isn’t really filling me with confidence that any of his future articles are going to be an enjoyable read, from a Canucks fan’s perspective.

  • DJ_44

    So, if I am to understand this article it is basically don’t blame me; I am just objective.

    You missed the point of many of the comments on the Gudbranson article. The critique was not that the video analysis was not objective (even though it obviously was not — tldr; see last sentence– ); the video analysis, specifically the clips presented and your the descriptions of what was occurring, was wrong.

    If it were a proper hatchet job (and one can do that on any player in league) you would select clips, properly point out deficiencies and mistakes, without the any sort of recognition of the other side of the coin.

    Your recent article did not even come close to that bar; it demonstrated a fundamental lack of understanding of the game. Specifically it mis-read what was occurring in the play, and then attributed actions to your perceived deficiencies in a player.

    In short; if you are claiming deficiencies in a player by showing clips that do not illustrate what you claim, how can your analysis be considered remotely objective?

  • Dirk22

    Bo Horvat struggles defensively. Please do an objective video analysis of why this might be the case. That might ease some of the persecution complex on these boards.

  • Laxbruh15

    “if we are proven right, there is nothing wrong with wrapping that up in a one-sided piece that focuses on nothing but examples of bad plays.” There is a significant issue with this. This is the flaw that continuously gets implemented in almost all of the content that’s produced on this site. Do you have any understanding of what the ethical code is for journalism? This violates the vast majority of all of its tenets. You have an obligation to provide balanced analysis which means that the amount of positive and negative content is supposed to be proportionate to the amount that actually occurred. Essentially, this article admits that this content has next to no legitimacy whatsoever. The blatant childishness is getting to be a little much.

  • tyhee

    Mr. Beichler, for the most part I agree with you, but I’m among the minority. You’re making your points to a group that by and large doesn’t and never will agree with you. I hope that doesn’t deter you from your analysis, which I find informative.

    I’m guessing that most of your readers are going to notice, when watching a hockey game, whether a player skates well, is hard on the puck, is tough, is good one on one and his good shots and pretty passes. Those things matter, but things like consistently making good small decisions about where to go and what to do are harder to notice at the rink (one would have to be concentrating on analyzing them, instead of, as most of us will do, simply watching and enjoying the play) and pretty much impossible to consistently track on tv. Your examples in analyzing Gudbranson’s play were mostly based on positioning and decision making-things that I don’t think most fans emphasize in forming their opinions about value of players.

    One thing (and I’ve mentioned it before) that I’m curious about is whether it is possible to quantify the value toughness brings to a team. It wouldn’t be easy as so much would depend on what deterrence value other teammates have so whether the team is otherwise short of it. The Canucks are short of toughness and with Dorsett out, it seems that Gudbranson is very likely to have value beyond his hockey decisions and ability. What value does deterrence have to a Canucks team that may otherwise be soft? Does Gudbranson lower the number of injuries to his teammates in a season?

    How much is toughness worth in today’s NHL? Presumably it is worth less today than it was during the heyday of the Broad Street Bullies in the 70’s, but how much less? That’s something I’ve never seen someone try to objectively quantify.

      • tyhee

        Thanks for the reply and the reference to the article.

        I think the methodology makes the article of little use, though.

        Stimson based his conclusions on the following:

        1. 1 chosen anecdotal example with video. Obviously one example doesn’t prove any point at all.
        2. Hit rates in general when 4 chosen players joined and left teams.

        There are two problems with this.

        The first is that hitting doesn’t equal making dirty runs at players, nor intimidating them. A hit is any body check when the opposition player has the puck. It doesn’t account for slashes, spears, hits on players who don’t have the puck nor distinguish between clean checks and dangerous ones.

        That in itself makes it of next to no value in determining deterrence to me. I’m not concerned about deterring clean play-I’m concerned with intimidation and injury-inducing behaviour.

        The other thing is that even with hits, in the examples Stimson used, the hit rates against the teams dropped noticeably with two players (Glass and Scott) on the teams, made no difference with another (Rinaldo) and the hit rate against Lucic’s team was actually higher when he was on the team than when not. Small sample size, two examples the hit rate goes up, one it goes down, one it stays the same. The total is an increase.

        So even using the flawed method of counting hit rates, with too small a sample size to make a lot of difference, there seems to be a small increase in hits against when the tough guy was off the team.

        Stimson’s conclusion may or may not be right, but I don’t see that he’s actually proven anything at all.

      • Canuck4Life20

        So typical. An incomplete analysis that provides weak evidence to support the position, but the conclusion is still reached that no further investigation needs to be done. This is why so many are writing off the advanced stats on this site. It’s not because they don’t have merit, it’s because they are generally used in selective and incomplete analysis by a group that clearly has very little experience or training in statistical analysis.

        But you keep banging the drum in your echo chamber. It’s much easier to tune out the criticism that way than it is to actually produce quality and complete analysis that will stand up to scrutiny.

      • RobG

        So you base your conclusion off one report that utilizes incomplete and insufficient information and say “thats it, theres the answer, no point and going further”

        You might as well put down your pen and go home son, you just lost all your credibility.

  • Jabs

    Analytics is just a made up industry for out of work computer geeks. Great work guys on getting some people to believe you but those of us who really understand the game and have played it at high levels and been fans of the game all of out lives know otherwise.

  • LTFan

    I am a regular reader of CA. My comment on some of the bloggers is that they have formed an opinion that a player or several players are not good. The blogger uses stats to back up his argument. However, as Janik Beichler points out: “We don’t want to include plays we might not have picked up as negative had they been made by a different player but use them anyway because they make this player look bad.”

    In the previous game against St. Louis a commentator pointed out that Gudbranson had excellent stats but were not mentioned or in the story written by J.D. Burke. In this case the stats. would have made Gudbranson look good.

    Anyway, I enjoy most of the articles and most of the commentators are good and a few are very good. What I find as offensive is when the commentator disagrees with someone and adds some cheap shot on a personal level somewhere in his criticism.

  • Holly Wood

    Getting tired of reading the long winded journalist types on here with no actual hockey playing experience try to thrust their version upon us using their analytics to drive their point. I would say for the most part they are educated, well spoken and accomplished writers, bloggers etc but their general knowledge of the game is sorely lacking.

        • Whackanuck

          Guys, analysis wasn’t invented by a bunch of fans sitting around a bar table. People who DID play, often at a high level were among the first to use them (Jim Corsi for example). Criticize the wannabe statisticians and certain writers on this board justly all you want but to totally deny the validity of hockey statistics is ostrichian. Many of the statistics are intuitiveley observable to very experienced hockey people but not even ALL ex hockey players. The whole argument is tiresome. Stick to the point…stats show aspects a biased eye misses. What seems real is not always so. That’s why coaches go over video. However,if one of you critics want to sit down and extract a dozen clips of great plays by Gudbranson and write an interpretive article and post it as a guest I’m sure you (and Gudbranson) would earn some cred. Or not.

    • Locust

      Holly Wood – it is beyond obvious that “the games” are not what’s important to most of the writers here. They can base their “points of view” and make their arguments based on the ‘statistical analysis’ of the game. Blatantly obvious earlier this month that a couple of writers never watched the game they wrote about.

      That is why so many people like you and me butt heads with them. Yet they continue to try and tell people that really know hockey – how it is. That WE are wrong.

      I’m sure they had great careers in grades 6-8 playing road hockey……