Rhys Jessop’s 2014 article, “We think the Vancouver Canucks may have a scouting problem(!!!!),” which replaced the Canucks’ scouting staff with the ill-prepared intern Sham Sharron, is a classic piece of hockey bloggery.

Jessop’s Sham Sharron thought experiment has been oft-cited and even repeated, most recently this past week in response to Jessop winning the Stanley Cup as a member of the Carolina Hurricanes’ front office.

It’s also fundamentally flawed, and it’s been bugging me for over a decade.

Sham Sharron is a cheater

The Sham Sharron thought experiment was a simple way to show just how bad the Vancouver Canucks’ scouting and drafting had been.

“I’ve put a summer intern with nothing but a book of CHL stats and no access to any non-Canadian junior league up against an entire team of world-travelling, game-watching professional amateur scouts,” said Jessop. “If the Canucks’ brass can’t clear this woefully low hurdle, then holy hell, they are awful.”

He laid out the rules for this intern, whom he named Sham Sharron:

All players selected will be from the Canadian Hockey League.
Goalies are voodoo, they will not be selected at any time.
Defencemen are voodoo, they will not be selected at any time.
The Canucks’ selection will be the player still on the draft board that scored the most points in their 17-year-old CHL season that was for real taken between Vancouver’s selection and Vancouver’s subsequent selection.
No other information other than the total number of points a player had in his 17-year-old season (his first year of draft eligibility) is considered. This information was freely available at the time each draft was held.
Ties are broken on the basis of points per game.

The idea was to remove the role of scouting and go entirely by the numbers, and not even a fancy analytical model, but the most basic boxcar numbers available: points.

Here’s the issue: point four means that scouts are 100 percent still in the picture.

This intern doesn’t just have a book of CHL statistics; he has the foreknowledge of how every single other NHL team is going to draft.

Some have characterized Sham Sharron as being simply about picking the highest-scoring player, but that’s not quite right. Instead of just picking the highest-scoring forward from the CHL, Sham picks the highest-scoring forward from among a select group of prospects: the ones that the scouting staffs of every other NHL team have determined are worth selecting in the vicinity of that draft pick.

Sham isn’t independent of the scouts at all. In other words, he’s cheating.

To see what I mean, let’s take a look at the very first Sham Sharron article.

2000 NHL Entry Draft

Jessop started his thought experiment with the 2000 NHL Entry Draft, and it immediately provides an example of what I’m talking about.

Instead of Nathan Smith, the original Sham Sharron hits a home run with Justin Williams at 23rd overall. But it’s immediately apparent from the chart that Williams is not actually the highest-scoring CHL forward available, because Sham selects two players who outscored Williams with his picks in the third round: Ramzi Abid and Michel Ouellet.

In other words, Sham comes to the drafting table armed with the knowledge that NHL scouts — specifically those for the Philadelphia Flyers, who picked Justin Williams 28th overall — see Williams as a prospect worth drafting in the first round.

Who does Sham Sharron select if he doesn’t have this information?

Well, setting aside Abid, who was actually a draft re-entry in 2000 after he was a second-round pick in 1998, the highest-scoring first-time draft-eligible CHL forward available was Carl Mallette.

Mallette had a fantastic draft year with the Victoriaville Tigres in the QMJHL, racking up 49 goals and 125 points in 69 games. He even played a premium position as a right-handed centre. Sham Sharron, without the benefit of any scouting reports, should be all over Mallette.

But NHL scouts assessed Mallette as having a low likelihood of reaching the NHL despite his prolific scoring in junior, and he didn’t get picked until the fourth round in the 2000 draft, when he was selected 107th overall by the Atlanta Thrashers. They were quite right in that assessment, as Mallette wasn’t able to stick in the AHL, let alone make the NHL.

Here’s how the rest of the non-precognizant Sham Sharron’s picks shake out:

With legitimately zero help from scouts, Sham still hits on Michel Ouellet for a solid single, which is still better than the Canucks’ actual 2000 draft class. But without the Justin Williams home run, it’s not quite as impressive, as he takes several players who ultimately, and justifiably, went undrafted.

Let’s take a look at a couple of other draft years tackled by Sham Sharron.

2001 NHL Entry Draft

The original Sham gets another great pick in 2001, snagging Jason Pominville in the first round, but only because he’s got some help from the Buffalo Sabres’ scouts, who decided he was worth picking 55th overall in the second round.

Without that knowledge, Sham would instead take Kyle Wellwood much sooner, picking him 16th overall, because how could he possibly know that Pominville is a more promising prospect than Wellwood when Wellwood has more points? He has no idea that scouts had such a dim view of Wellwood’s future, with the Toronto Maple Leafs not drafting him until the fifth round.

Honestly, it’s still a fine pick by Sham, as Wellwood goes on to play more NHL games than 17 of the players actually picked in the first round, but taking him so early and losing out on Pominville hurts the draft class as a whole.

Missing out on P.A. Parenteau at the end of the draft also speaks to the flaw in the original Sham Sharron. Sham gets Parenteau because he gets to pick from the 44 players selected by other teams after the Canucks’ pick at 245th overall, benefiting from the scouting work done by all of those other teams.

Without that knowledge, Sham instead goes with the undrafted Martin St. Pierre, who still got a cup of coffee in the NHL, but didn’t have anywhere near the same impact.

2002 NHL Entry Draft

I have to note that the rules restricting Sham Sharron to forwards really hurt here, as he could have had defenceman Ian White, who outscored the vast majority of CHL forwards in his draft year. He didn’t get picked until the sixth round by the Toronto Maple Leafs on account of his 5’10” stature, but went on to play 503 NHL games.

Alas, Sham is restricted to picking forwards, which works out all right. He still gets Matt Stajan and Maxime Talbot for a solid showing compared to the Canucks’ scouting staff.

But things go wrong the following year.

2003 NHL Entry Draft

Oof.

The first round of the 2003 draft is one of the best in NHL history, but, armed with just the numbers out of the CHL, the non-precognizant Sham Sharron goes way off the board and takes Nigel Dawes instead of Mike Richards, because Dawes outscored Richards by 10 goals and 5 points.

In real life, Dawes didn’t get picked until the fifth round by the New York Rangers. Considering he went on to play 212 NHL games — more than any forward taken in the fourth round that year — does show that he should have been picked sooner.

Without the help of any scouts, Sharron also doesn’t find Clarke MacArthur or Brad Richardson, leaving the 2003 draft with just one kinda sorta NHLer.

The point of Sham Sharron

I could continue, but I don’t want to belabour the point. Besides, the overarching meaning of the article — that the Canucks’ scouting and drafting was woefully bad for over a decade — is still correct.

Heck, both versions of Sham Sharron draft Claude Giroux over Michael Grabner in 2006, because of course they do. It was, indeed, dead simple to predict that the 103-point Giroux was going to have a better NHL career than the 50-point Grabner, though many teams besides the Canucks misjudged Giroux, who never should have fallen to 22nd overall that year.

In addition, the work that came out of Sham Sharron — analytical models and more in-depth research into how to make drafting better — have been incredibly valuable, with several of the people who did that work winding up working for NHL teams as a result.

But it’s still important to note that Sham Sharron depends on a foundation of scouting, using the pool of players that were drafted in the vicinity of the Canucks’ picks, then skimming the cream off the top.

The actual lesson from the Sham Sharron thought experiment is that scouting and analytics should not be opposed to one another, but should work in concert to provide the best results.

That’s a conclusion that Jessop himself would presumably agree with. After all, he works for the Stanley Cup Champion Carolina Hurricanes as a scout.