Decision Making Bias: Survivorship

Learn and practice avoiding Survivorship Bias with exercises.

What is Survivorship Bias?

The concept in one sentence:

Only paying attention to those who survive or succeed can mislead you.

The concept in one quote:

Survivorship bias can become especially pernicious when you become a member of the ‘winning’ team. Even if your success stems from pure coincidence, you’ll discover similarities with other winners and be tempted to mark these as ‘success factors’. However, if you ever visit the graveyard of failed individuals and companies, you will realize that its tenants possessed many of the same traits that characterize your success.

Rolf Dobelli

The concept in one image:

How I use it?

In order to better understand success, I look for the differences between what succeeded and what failed.


Examples

Example 1

The survival rate for a disease is pretty high.

Can an unhealthy person be confident that they will survive?

  • Yes

  • No

Answer:

The correct answer is No.

The survival rate could be high for healthy people and low for unhealthy people.

Example 2

A commonly held opinion is that music from previous generations is better.

Is it true?

  • Yes

  • No

  • Neither, It's subjective.

Answer:

The correct answer is No (but neither is fine too, people have different tastes).

The best music from previous generations survived and the bad music died out. This means that people are comparing the best music from previous generations with the current music of today (good and bad).


Exercises

Exercise 1

Bill Gates, Mark Zuckerberg, Michael Dell, Steve Jobs, Ralph Lauren, and Gabe Newell dropped out of college and later became billionaires. Inspired by this pattern, your cousin wants to drop out of college and launch his own company.

Should your cousin drop out?

  • Yes

  • No

Exercise 2

The army wants to reduce aircraft casualties by reinforcing certain parts. While studying returning aircraft (those who came back safely), they see the following:

  • Wings - 80 bullet holes on average

  • Tail - 40 bullet holes on average

  • Engine - 5 bullet holes on average

Which area of the aircraft should they reinforce?

  • Wings

  • Tail

  • Engine

Exercise 3

The top 3 designers in the world went to the same design school.

Is that enough information to apply to the design school?

  • Yes

  • No

Exercise 4

You saw the following advert on Facebook:

"In just 6 weeks you can look like this if you join our gym class. We have more than 100 success stories on our website!"

After going through the success stories (with before and after pictures) on their website, you are thinking of joining the class.

Did you have enough information to make a decision?

  • Yes

  • No

Exercise 5

After issuing helmets during World War I, the number of head injuries rose dramatically.

What happened?

  • Bad helmet design.

  • More survivors.

Exercise 6

Over 5,000 different people have climbed Mount Everest and reached the summit.

Is it safe?

  • Yes

  • No

Exercise 7

You want to study some nonprofit organizations in order to be better prepared to start your own.

What should you focus on?

  • Successful nonprofits

  • Unsuccessful nonprofits

  • Both (successful and unsuccessful)

Exercise 8

A commonly held opinion is that tools from previous generations are better built.

Is it true?

  • Yes

  • No

Exercise 9

A financial report on the performance of U.S. companies in 2020 doesn't include any company that closed down that year.

Can you trust this report?

  • Yes

  • No

Exercise 10

A company did a job satisfaction survey with its current employees. The management team plans to use the results to make changes.

Did the management team fall prey to the "Survivorship Bias"?

  • Yes

  • No


Answers

Exercise 1:

The correct answer is No.

Your cousin is not taking into account the number of people who followed this pattern and failed.

This scenario seems oddly familiar…

Exercise 2:

The correct answer is Engine.

In this scenario, you have to think of the aircraft that survived and those that didn't. Those that survived came back despite the high amount of bullet holes on the wings and tail. But not a lot of them came back with bullet holes on the engine. Either the army's enemies kept missing or damage to the engine was deadly.

Fun fact: This was a real scenario.

Exercise 3:

The correct answer is No.

We have to know about the other student's performance (grade or achievement). The top 3 designers could have been an exception.

Imagine if the top 3 designers graduated in 1991, and since then the school only had bad performances.

Exercise 4:

The correct answer is No.

You have to know how many people failed vs how many people succeeded.

Exercise 5:

The correct answer is More survivors.

A lot of the soldiers that would have died from head injuries, survived due to the helmet’s protection.

Another fun fact! This was a real scenario too.

Exercise 6:

The correct answer is No.

We don’t have enough information to know if Mount Everest is safe. The scenario only gives us the number of people who successfully reached the summit. But we have no information on those who failed and the risks involved.

Morbid fact: More than 300 people have died attempting to reach the summit. The only years without known death were 1977 and 2020 (due to coronavirus restrictions).

Exercise 7:

The correct answer is Both (successful and unsuccessful).

You have to know the differences between successful and nonsuccessful nonprofits.
Without knowing the differences, you won’t know if the “thing” that a successful nonprofit did was the reason for its success.

The “thing” could be a marketing campaign, a process, a project, etc.

Exercise 8:

The correct answer is No.

The best tools from previous generations survived and the bad tools died out. This means that people are comparing the best tools from previous generations with the current tools of today (good and bad).

Example: Baby Cage, popular in the 1930s.
How to provide babies with fresh air and sunshine when you live in a crowded city.

Exercise 9:

The correct answer is No.

Since the report only includes companies that survived, the performance will be higher.

Exercise 10:

The correct answer is Yes.

The survey doesn't take into account ex-employees and why they left.


Messages and feedback are welcome via email.


Bonus

A relevant comic from xkcd.