Survivorship Bias: Why these deceptions undermine our language teaching and 5 tips to annihilate them

Introduction

We might be wondering: how is something like “SURVIVORSHIP BIAS” relevant to language educators? Apart from the inherent thirst for knowledge, this bias has a far greater influence on our approaches to language teaching than we realise. It can creep into our evaluations of strategies and the beliefs we uphold regarding the factors that contribute to effective language learning.

To combat survivorship bias and foster a more realistic approach to language teaching, there is a need for awareness and effective strategies. In this article, we will explore 5 tips to annihilate the deceptions caused by SURVIVORSHIP BIAS and ensure a more accurate understanding of language teaching.

What is SURVIVORSHIP BIAS?

SURVIVORSHIP BIAS occurs when we focus on the entities or individuals that endured or passed through a selection process (e.g. an adverse event, a screening exercise), while inadvertently overlook those that did not make it through. This cognitive bias tends to lead to unrealistic expectations, misguided strategies, and/or incorrect conclusions because it fails to take into account all the relevant information. We have the tendency to encounter and demonstrate this bias due to the following reasons:

• Gathering data from failures may pose a greater challenge, if not completely impossible;
• Our desire to find hope in stories of survival and success;
• Role models provide inspiration for a cause despite being the anomalies;
• Media coverage often features extraordinary individuals rather than the common folk;
• We tend to emphasise our more positive qualities in public, rather than putting our darker aspects on public display; and
• We have a skewed representation of probability and may conflate correlation with causation.

As such, despite the potential learnings that can be garnered from failures, we often find ourselves immersed in the misleading narratives of triumph.

SURVIVORSHIP BIAS Examples: the classic WW2 plane story and more

The textbook example of SURVIVORSHIP BIAS was a study in which a brilliant mathematician, Abraham Wald, was involved. In this classic story as premised within World War Two, the Allies sought the expertise of a group of researchers to determine where additional armour should be placed on airplanes to bolster their durability without overweighing the plane.

The military officers provided statistics that were apparently useful, displaying the distribution of damage on planes that returned from battle (aka the survivor planes). These planes revealed bullet holes primarily in the wings, tail, and body. As such, the military officers’ initial proposal was to reinforce these heavily damaged areas with armour.

Yet, Wald astutely discerned that these were not the areas to focus on since these planes managed to make it back despite the damage. He argued that it was the planes that did not return from battle, those that didn’t survive, that should dictate where to strengthen armour – presumably the engine (the fourth data point collected).

Wald’s discovery demonstrated the dangers of the SURVIVORSHIP BIAS (presumably a term that he coined). Should the military adopted the original proposal, it could have implied more casualties and losses for the Allies.

Now of course, falling prey to SURVIVORSHIP BIAS is not something restricted to life and death issues. Other famous examples include:

• Entrepreneurships, startups and businesses: Many hail icons like Steve Jobs, Mark Zuckerberg, and Bill Gates as symbols of entrepreneurial success, mistakenly assuming their common traits of being a white male college dropout is the key to become successful entrepreneurs. This overlooks the numerous entrepreneurs who might also be similar but faced failures, nevertheless.
• Finance and investment: The long-term performance of mutual funds and stocks may be wrongly understood if the evaluation hinges solely upon extant data (aka the surviving funds), neglecting the inclusion of underperforming funds that no longer exist.
• Healthcare: In calculation of survival rates after disease diagnoses, healthier patients represent a larger proportion of the sample cause the results to be skewed. During the COVID-19 pandemic, experts were also cautious in trusting institutional calculation of survival rates, as patients who die without testing are not able to be included in the data.

SURVIVORSHIP BIAS in language education

Now we may wonder: how is the SURVIVORSHIP BIAS relevant to language education, and thus us – the language educators? During a recent discussion with a fellow researcher with whom I collaborate closely, we engaged in a discussion concerning the perennial challenge of enabling students to successfully learn their second language.

Her primary aspiration is thus to develop a cohesive array of strategies tailored to support these students in their language learning endeavours. In pursuit of this objective, she wanted to propose a formal research project which investigate the practices of accomplished language learners, with the ultimate aim of formulating a well-founded set of strategies.

Does that not remind us of the earlier examples? Also, we may also have encountered inspiring examples of language learning – those that have gone against all odds to become established polyglots. We might have even taught them in our own career or interacted with them during conferences and seminars. Personally, I just attended a conference last month where a panel of successful language learners were sharing their experiences with an audience of language educators.

The proliferation of success stories in language learning can be attributed to SURVIVORSHIP BIAS because those who have succeeded tend to be more vocal and share their experiences. Not to mention the many “gurus” out there who may be sharing the “state-of-the-art approaches to language learning” and provide a whole list of testimonials of their success. People are naturally drawn to such successful examples, as they provide inspiration and a sense of possibility. 

However, by only focusing on these success stories, we overlook the vast majority who have faced challenges and setbacks in their language learning journeys (or the negative testimonials that would never make it to a marketing piece). This creates an unrealistic perspective that undermines the efforts of those who are not experiencing the same level of success. In turn, it can lead to feelings of frustration, self-doubt, and potentially, abandonment of language learning goals. So, how should we deal with such encounters?

1. Understanding SURVIVORSHIP BIAS as the culprit leading us to unchecked success examples.

By reading this article, we are already one step ahead in our combat against this bias. Staying aware of SURVIVORSHIP BIAS enables us to be more mindful when listening or reading language learning examples of success.

We must first recognise that the success stories we encounter may not represent the reality for all learners. By acknowledging the limitations of success stories, we can then strive to gather a more balanced and comprehensive understanding of language learning.

Adding another point, we can even encourage open and honest discussions about failures and setbacks, highlighting that these are integral parts of the learning process and should not be seen as obstacles but rather as opportunities for growth and improvement.

By fostering a growth mindset and a culture that values effort, resilience, and learning from mistakes, we can help our learners navigate the language learning journey with a realistic and determined mindset, ultimately leading to greater success and fulfillment.

2. Prevent SURVIVORSHIP BIAS by considering counterfactuals.

At this juncture, I must highlight that I am not against the sharing of success stories. The recognition of SURVIVORSHIP BIAS is the enlightenment that the narrative of an issue is incomplete with only the elements of success.

This is why we should consider counterfactuals. Now, to be explicit, counterfactuals does not necessarily refer to unsuccessful learning examples specifically even though they can be taken as evidence for counterfactuals. The crux is that when we derive “insights” from success stories, we should exercise the habit of contemplating whether the same “insights” hold the potential to yield contrasting outcomes – thus failures.

By way of example, when we learn from success stories that employing flashcards for self-directed learning yields excellent results in language learning, we can cultivate the habit of imagining scenarios and potential factors where the use of flashcards did not yield comparable outcomes.

For validation, we try to confirm the presence of actual instances of this counterfactual and assess its plausibility (e.g., the likelihood of its occurrence being high). This provides us with a safeguard against accepting the insights from the success story uncritically.

3. Avoid SURVIVORSHIP BIAS by widening our perspectives.

If we revisit the original story that gave us the concept of SURVIVORSHIP BIAS (aka the WW2 plane study), we should understand that the root of the issue lies in incomplete data (inadequacy of the sample): the military were not able to see the whole population of airplanes – all surviving and perished airplanes. It was extremely challenging, if not nearly impossible, for the military to recover the sample subset that perished. 

Luckily for us, diversifying our sample is largely achievable. By expanding the number of observations (or learners in our particular context) across various categories, we are then better able to accurately depict the actual population (e.g. aligned with the profile of learners that we encounter) and assess the effectiveness of specific approaches or strategies for them, as well as their efficacy under specific conditions.

Widening perspectives is also about recognising the individual differences that are influential in explaining variance in language learning outcomes. To put it simply, language learning is a highly individual process, influenced by various factors such as motivation, personality and beliefs. We probably have heard this very often: “What works for one person may not work for another.”

What does this really mean? This means that we should widen our repertoire of language teaching approaches (methods and techniques included), and not solely rely on those that success stories have depicted. We may probe and ask: if we cannot do all, then how do we choose one over another?

An informed consumption of research would be crucial here. Taking reference from the learner profiles with whom we are dealing, we look for bodies of research that have provided substantial evidence that tease out effective approaches while controlling for individual differences. Before we are able to acquire such knowledge, we ought to remember that widening our perspectives helps us avoid SURVIVORSHIP BIAS and encourages a more inclusive and adaptable approach to language learning.

4. Maintain a critical mindset to question assumptions to sieve out SURVIVORSHIP BIAS.

If I were to highlight THE most critical guardrail from any bias in particular (perhaps this is also my personal bias?), it would be our own critical thinking. This involves our natural tendencies in processing information, whether it be factual data or differing viewpoints, gleaned from research or any media material.

By cultivating a critical mindset, we maintain the habits of questioning assumptions underpinning any information and critically evaluating their validity. This helps to sensitise us to biases, one of which is the SURVIVORSHIP BIAS, and pay attention to the possible missing details that should be part of a whole.

Do note that our learners can also fall prey to the SURVIVORSHIP BIAS much like us. In their case, this bias can cause them to interpret certain handpicked strategies as the definitive pathways to success. Their judgments may often be influenced by observing the achievements of their top-performing peers and by embracing commonly held beliefs that have been shaped by success stories – some of which belong to their own.

As such, we should also be mindful to incorporate the cultivation of critical thinking into our language lessons to help our learners see the bigger picture. When our learners evaluate their own strategies, they should pay attention to both their successful and not-so-successful attempts. This can help them tease out their better strategies in the longer term.

5. Use classic examples of SURVIVORSHIP BIAS in business, sports and entertainment to educate.

The classic examples (especially the WW2 plane study) of SURVIVORSHIP BIAS are evergreen reminders to sustain our awareness of its existence. Keeping these examples in mind is a measure we can actively take to keep the bias away.

Despite so, we might just want to compile a set of different examples that might better cater to our students’ interest and cognitive level. The previous instances from business, finance, and healthcare might not be as captivating or relatable to them. Instead, examples derived from the sports and entertainment might resonate more effectively with them.

To our advantage, SURVIVORSHIP BIAS is everywhere we look and talk about in sports and entertainment. More often than not, we idolise the star athletes and hear stories of their triumphs but fail to consider the countless individuals who never made it to the professional level despite their talent and dedication. We also celebrate the achievements of famous actors, musicians, and artists, without acknowledging the vast number of individuals who struggle to make a living in such a competitive field.

Drawing on such examples, we can draw parallels for our learners to re-evaluate their representation of the factors leading to success in language learning. This supports our learners in developing a more comprehensive and inclusive approach to language education that acknowledges the struggles and provides adequate assistance and opportunities for all learners to thrive.

Making a difference in language education, one sip at a time. Support my work with a coffee?

Conclusion: overcome SURVIVORSHIP BIAS by going beyond our role models.

SURVIVORSHIP BIAS can pervade our lives, in the research and media content that we consume frequently. We must recognise that not all successful individuals are representative of the entire population, as survivorship bias often leads to an unrealistic perception of the likelihood of success.

On a related note, it’s important to recognise how our inclination in seeking out role models can also undermine our language teaching methods and hinder our students’ progress. By following the 5 tips I provided, I do hope we can all actively work towards annihilating SURVIVORSHIP BIAS in our language education practices.

Thank you for reading! If you like what you are reading, do subscribe to our mailing list to receive updated resources and tips for language educators. Please also feel free to provide us any feedback or suggestions on content that you would like covered.