The Impact of Cognitive Biases on our Evaluation of Play

The other day I listened to an audio file featuring Panthers head coach Gerrard Gallant and assistant coach Mike Kelly. They were asked about the emergence of modern statistics in hockey and their consideration for that data. What ensued was a candid discussion that made for an extremely frustrating listen. Here are some direct quotes…

“When we get the sheet that’s got the guys who are the best offensive players, the best defensive players, the best faceoff guys blah blah blah – about five different things… we knew it all before we got the sheet”

“If you do your work and you’ve been around the league, you’ll know the players around the league.”

“One of our players was our top analytics player and I couldn’t stand watching him on the ice. He was our top analytics player and I didn’t like the way he played one bit. So some of it is really good, but some of it you can’t get fooled by it”

That last quote (which seems to be referring to ex-Panther Sean Bergenheim) is the most troubling. It’s a perfect example of searching for and interpreting information in a way that confirms one’s existing beliefs. You may or may not recognize this an example of confirmation bias – a treacherous cognitive bias that can lead us into making poor decisions by ignoring valid, yet contrary evidence. Confirmation bias is just one of the numerous cognitive biases that can steer us away from rationality.

I wonder if Gallant and Kelly’s tones would change with respect to modern statistics if they were to gain a better understanding of how our brains process large amounts of information. Because that’s what we do when we base our analysis of hockey on observations alone – we attempt to process a very large number of events. We like to think that our beliefs are derived from the objective analysis of our accurate and expansive query of observations. The prevalence of confirmation bias proves that we aren’t as objective as we think we are and reading further will demonstrate how troubles also arise when sifting through our expansive memory banks.

In addition to biases we have heuristics. A heuristic is a mental shortcut which helps speed up decision making. Without heuristics we would spend most of our time overanalyzing problems before making a decision, leaving us with very little time to react. The issue is that sometimes these shortcuts can lead us into making irrational decisions – that’s when they take their place alongside confirmation bias as cognitive biases.

We feel safer driving in a car than flying in a plane. Why? Because we tend to overestimate the likelihood of events that stand out in our minds – the catastrophic event of a plane crash fits that description.

This is an example of availability heuristic – the tendency to overestimate the likelihood of events with greater “availability” in memory, which can be influenced by how recent those memories are or how emotionally charged they may be.

According to the National Highway Traffic Safety Administration, there are approximately 1.27 deaths per 100 million miles travelled by car in the United States.  That number is virtually 0 per 100 million miles travelled by U.S. air carrier (a statistic compiled by the National Transportation Safety Board). Based on these relevant statistics, it’s apparent that flying is the far safer mode of transportation.

The funny thing is that even after reading those statistics, the next time you board a plane you will still likely feel a higher sense of anxiety than the next time you start your car and shift into drive. That’s just how powerful availability heuristic is.

When we evaluate the play of individuals in hockey, we sometimes fall victim to availability heuristic. There’s a reason why NHL coaches feel safer deploying defensemen like Roman Polak when defending a lead than say, Jake Gardiner. Jake Gardiner has been known to give the puck away, pivot in a lazy fashion, lose his man and occasionally the result has been a goal against. Many would conclude that he is prone to mental lapses in the defensive zone and it seems safe to assume that the Leafs give up more chances against with him on the ice than a “safer” alternative like Roman Polak. Scoring chance data (via War-On-Ice.com) from this past season suggests that the opposite is true – Jake Gardiner SCA/60: 28.5 Roman Polak SCA/60: 31.7

In hockey, mistakes that directly lead to quality chances against are more available in memory than the small errors like needlessly chipping the puck into the neutral zone, passive zone entry defense etc which are also linked, albeit less directly, to conceding scoring chances against. Gardiner’s errors are more dramatic in our memory than Polak’s in the same way a plane crash stands out in our minds more than a car crash.

Availability heuristic is actually the phenomenon Tyler Dellow was describing in this quote from his well-known “Big Mistake” piece…

“If someone asked me what I think the biggest failing of the eyeball test is, I’d respond that it’s the emphasis on the big mistake. There are gigabytes of information contained in a hockey game. So much information that I think it’s difficult for anyone to take it in and organize it rationally. The way that our brains deal with that is by focusing on the big mistake.”

Clearly there are some major issues with evaluating play by relying solely on observations. Studies have shown that even improper linear models (models developed by estimating the weights of predictive variables) are better predictors than human judgement. It’s apparent that implementing valid and reliable data as part of the decision making process can help us overcome some of the biases and heuristics that actively cause us to deviate away from logical interpretation and accurate judgement. The ignorance and arrogance portrayed by Florida’s coaching staff can only hamper their team’s chances of success in an environment as competitive and unforgiving as the National Hockey League. I’ll leave you with this…

“We can be blind to the obvious, and we are also blind to our blindness.” – Daniel Kahneman, Thinking Fast and Slow

Advertisements

5 thoughts on “The Impact of Cognitive Biases on our Evaluation of Play

  1. I really like the heuristics angle, instead of coming across as ‘your thinking is basically and consistently flawed’ it recognizes that it’s an efficient way of thinking that isn’t accurate enough to count on for something specific and high stakes as building an NHL team.

    …I wonder if big mistake players might be making them because they’re avoiding constant small mistakes, it really seems to fit (heuristically) well with the example of Gardiner making visible mistakes but being better on overall chances.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s