The Dunning-Kruger Effect: A Rite of Passage

The Dunning-Kreuger effect can be a sore subject for a lot of folks but understanding it and accepting self-awareness is a key to moving beyond the barrier of the Peter Principle (And to succeed in a high-growth organization). Heck, I still feel the sting of embarrassment when I think of my time as an over-confident and budding leader. Co-authoring this article with me is Quan Heng Lim, the Regional Head of Privasec in Asia.

We worked together for quite a while at Horangi and faced a variety of challenges with our customers, our team, and even ourselves. For myself, Quan Heng brings an interesting perspective of my own leadership journey as he was my Chief of Staff. If I’m honest, it can still be a bit painful when we reflect on some of the classic leadership challenges we fumbled through — however, we both welcome the discussion and perspective as it’s the best way to grow.

I’ll let Quan Heng take it away from here and I’ll pop back in with some of my own anecdotes and personal perspectives.

Quan Heng Lim - Chief Delivery Officer @ Privasec

What is the Dunning-Kruger effect?

“One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision.” — Bertrand Russell

The Dunning-Kruger effect describes a psychological bias, where individuals with low ability at a task perceive their ability to be higher than it actually is, while those with high ability underestimate theirs due to awareness of their own limitations. The original study focused on 4 key areas but its principles have been applied to various other contexts.

This idea is so intuitively relatable it’s been misappropriated in many ways.

The interpretations vary, but many have come up with their own names and forms around related concepts — Four Stages of Competence, The Peter Principle (As Lee mentioned above), on the flip side, the Imposter Syndrome — showing just how many have tried to shine the light on this commonly seen and impactful effect.

Let’s jump into our experiences with this stumbling block that so many of us will, or have, experienced.

*The original text can be found here, free to access.

Setting the Stage

Lee — During my time as a CTO, I was entrusted with overseeing both the engineering and consulting teams. We attracted some amazing talent, built some awesome cyber security products, developed some evasion techniques for a not-so-happy EDR vendor, and did some really amazing research.

We helped build the cyber security start-up community. We set a precedent for cyber security in the region. And we did it as a team. Horangi is a world-class organization still doing world-class things. For me, our Tigers (Horangi vernacular for team members) taught me how to be a better leader.

Q — It’s always exciting to be part of a fast-moving organization, even more so when you’re in a position to influence its growth and lead a team of highly skilled and motivated people running the same race. Tactical and operational responsibilities evolved quickly, to support rapid growth and expansion — and we all grew with it in an accelerated but enriching experience.

The Pride Before the Fall

Lee — Back when we founded the company I had a good dose of youthful energy and about 10 years in the cyber security sector under my belt. I took pride in my “cutting-edge” knowledge, which I considered an asset in a fast-paced industry like ours.

I was quite content in my bubble of self-assuredness, blissfully unaware that I was a walking example of the Dunning-Krueger effect. I believed that my prowess in cyber security and some experience with operating small side businesses would somehow make me the best start-up leader there ever was. I was pretty much at the peak of “mount stupid”.

As Horangi grew close to 100 team members, I noticed that the company seemed to be outgrowing my leadership ability. The good metrics were there, the team was rocking and rolling, but the dynamics were changing.

Instead of consulting with my mentors or doing a deep-dive analysis of myself, I jumped the gun. Convinced that I had my finger on the pulse of organizational behavior, I embarked on a grandiose restructuring plan. Departments were shuffled, people were promoted out of positions they were good at and enjoyed, and ideas for new titles were thrown around in my head like confetti at a New Year’s party.

Q — When I joined Horangi, I was confident in my technical skill set, in speaking the same language as anyone in the field, and ability to pick things up quickly to deal with challenges that would come up on the way. Working with a group of very motivated individuals and an environment that empowered initiative fueled that confidence, as we made quick progress.

As leaders and pioneers — especially those in a knowledge-driven field(such as technology/cybersecurity), we have all at some point relied on intelligence, skill, or just sheer determination to push through the challenges we face daily.

At that stage of our growth — if I were asked — can somebody with the skillset of a generalist manager do what we did, the answer would have been a resounding “No”. I simply could not see the other attributes needed ‘tomorrow’ in the rapidly growing team.

There are many allures of joining a high-growth organization, from career growth opportunities to the ability to make an impact and the dynamism to empower individuals, among other things. In my view then, an outcome of that momentum of growth is that there is always an observable bottleneck in the organization, with individuals moving around like a game of musical chairs.

We learnt at our own pace — the organization grew to take up the capabilities that were available, until the next bottleneck appeared, at operational, tactical, or strategic levels. I did not realize until after the fact, that this was the Dunning Kruger effect in full force, and it was hard to ‘hire’ the problem away. As Lee put it (Not specific to this example), ‘If a person doesn’t know what good looks like, how can he identify it in others?’

The Realization

Lee — The result came eventually — confusion spread and performance dipped on some teams. In hindsight, I realized that I’d skipped several necessary steps — consulting with other entrepreneurs, making gradual changes, reading through reference material, and above all, listening to my team.

I threw myself in the ocean and I was swimming through the valley of despair. The climb back to the surface was grueling and it was a “learning phase” of mixed metaphors that stings to this day.

Q — As the team and the scope of my responsibility grew, I struggled in moving from the role of a specialist manager, to that of a generalist.

In a young and growing team/organization, where there is a constant pressure on individuals to keep up with the growth of the company it can be present in both its leaders, those being groomed for management, and individuals in the team, and I could see that same pressure on the others around me.

There were times when conflicting feedback and information affecting decisions led to frustration, but also called out to me that the effect was clearly not just affecting myself. Over time, negative feedback, lack of progress and the loss of some of that efficiency and motivation in myself and the people around me got me thinking.

On hindsight, my key challenges and failures were fourfold, some of which were easier to address than others:

• With the pace of growth, I misjudged in thinking I had to quickly become a strong technical leader, vs moving away from the position of ‘having the facts’ towards other forms of leadership and enabling others;

  
  

• Gain the skills to identify the same issues in our young but rapidly growing team, especially those who appeared confident, or whose culture and communication style differed greatly.

  
  

• Help fellows in our knowledge-driven fields change their perception of what a leader needs to be, and when this would apply; and

  
  

• Help the rest of the team understand they were big fish in a small pond that would rapidly grow, especially those who were used to larger, slow-moving organizations.

  
  

The list goes on, but these are what will help the other budding leaders out there, especially those in knowledge-driven fields.

The Outcome

Lee — During one team off-site, Quan Heng made an enlightening observation — I was better at small team leadership than I was with larger teams. I hadn’t figured out how to lead leaders yet. While I started to learn from Quan Heng’s observations and started to grow again as a leader, I was about to learn the next lesson on delegation vs abdication — but we’ll save that story for another day.

I find it a bit ironic that I prided myself on my expertise in cyber security, yet failed to see the vulnerabilities in my own ability as a leader. Thankfully, life has a way of humbling even the most cocksure among us. I view this particular phase of my life as the tail end of my transition from an individual contributor to a Leader. For me, like many others, this transition was a team effort built on mutual trust — I will forever be grateful to our Horangi Tigers and the support they continue to give. And, while leading a high-growth team is never the smoothest ride, the bumps along the way have made me, dare I say, a little wiser.

Q — In some ways similar to Lee, the irony for myself was that, in the hectic growth, I failed to see these vulnerabilities in myself. I was confident in my technical capabilities, and my ability to learn to adapt — having not realized that I ‘didn’t know what I didn’t know. To compound this, I failed to intervene or take the actions that would have opened the door to learning from my mentors, and the peers around me.

Coming to the realization of how far the gap was then took time, and a lot of conscious effort to overcome and break out of old habits, (Not forgetting the swallowing of pride to accept). All that said, this struggle was one of the points in my life where I learnt many valuable lessons in such a short period of time.

Final Thoughts

These issues are not so easily identified or resolved in the thick of the action — I once asked another prominent leader in the organization early on how they would describe the organization’s view on its employees — and he replied ‘forgiveness’.

Forgiveness was part of the equation of understanding that further mentorship and growth are needed, the immediate goals and resources we had, building loyalty, versus the growth we might have had otherwise. Sound and objective judgment is needed, to call out cases where the Dunning Krueger effect was getting in the way of change for the better.

As we like to say, you don’t know what you don’t know. Knowledge of your biases, self-awareness, and reflection will bring us a long way. Mentors (To us, and ourselves to the people around us) can provide insights against the Dunning-Kruger bias by offering external perspectives on an individual’s actual competencies. Bridge the gap between perception and reality. Recognize when you’re no longer good at what you need to do, or when you need to change.

In an environment where everyone is learning and growing by the day, we also need to be careful of mixing up confidence for competence in the people around us — as many of us have been taught that confidence is one of the key factors to succeed. The prevailing idea of ‘faking it till you make it’ in high-growth environments, where people are constantly adapting and growing adds to the murkiness of this. The individual’s growth and the team around him will suffer from this lack of awareness.

The book “Traction” by Gino Wickman has a simple but effective framework for this called “GWC”. GWC is a simple test to get the right people in the right seats by identifying if a person “Gets the role, Wants the role, and has the capacity to perform in the role”.

Being in a high-growth organization is hard — there must be more we can do.

In my view, a homogenous team culture and identity has its strength in its agility (At the expense of moving far.) Building a team ‘In our image’ is also always a temptation. However, building “strong” would mean leveraging team diversity, and teammates that stay together and drive each other to learn.

A mix of backgrounds, expertise, and experience, which can counterbalance individual biases in perception and ensure well-rounded decision-making. It was these factors that allowed us to succeed through the challenges. We didn’t build a team of yes-people. We sought out new points of view and encouraged diversity of thought.

Not forgetting — The Imposter Syndrome

Recognizing the other extreme end of the spectrum is Impostor Syndrome — where individuals doubt their accomplishments and fear being exposed as a “fraud,” even if they are highly competent and successful. We’ll talk about the Imposter Syndrome more another time but it’s important to note helping leaders develop an accurate self-image enables them to seize opportunities they might not otherwise pursue. This is true on both ends of the spectrum.

I hope that this article will be helpful to others who have the privilege of being in a fast-growing organization, those who are current leaders, and those who have leadership opportunities in front of them. I will always be grateful to be part of the teams that fought through rapid growth, achieved that product-market fit, and made a real impact.

I am especially thankful to those few teams I’ve worked with who achieved the ultimate goal of becoming a “leadership factory”. Without these experiences, these demonstrations of “good enough”, and the influence of the leaders we worked with; we could not have grown as individuals, and could not have helped the other leaders at Horangi grow such an amazing organization.