In this article, we will explore how to evaluate a tipster using survivorship bias and also introduce a mathematical formula to quickly judge a tipster's abilities.

How to select a tipster using survivorship bias ?

Imagine there is a tipster who, over five years, has had a 100% success rate in predicting tennis matches with an equal probability of success, resulting in a $50,000 profit. Impressive, right? Not so much if you discover that this star tipster was actually a clown.

Let’s take the example of a simulation where 10,000 tennis tipsters (or clowns, it doesn’t really matter) each have a 50% chance of winning $10,000 per year or losing $10,000 per year. If a tipster has a losing year, they are eliminated.

The tipsters/clowns make their predictions simply by pressing one of two buttons. If we conduct the test for one year, 5,000 of our tipsters would have a profit of $10,000, while the same number would be down $10,000 and eliminated. In the second year, 2,500 clowns with a perfect record would remain, and if we continue like this, by the end of five years, there would be 313 clowns from the original group who, statistically, would be able, by pure chance, to make successive accurate predictions and win $50,000.

Confusing survivors with savants

This phenomenon is called survivorship bias, and it has significant implications in the real world of tipsters. The tipster currently topping the leaderboard on a betting site might just be a lucky clown pressing a button.

What are the important factors that influence this process?

The size of the initial sample is crucial. If you only focus on the winners in this process, ignoring all the other billions of clowns producing random results, you are being fooled by randomness. The simple fact is that by starting with a large enough sample, some participants will end up looking like savants purely by chance.

Another crucial factor is the probability of the event. Our example used a fair coin toss (50/50 chance for each side), but in the real world, a bookmaker has an advantage—this is the margin they make on each bet. Repeating our test with higher margins would produce fewer lucky winners: the lower the margin, the easier it is to achieve long-term success.

In a very simple way, a good indicator of a tipster's quality would be to check whether they use Pinnacle… or not! The odds from this bookmaker are proven to be the best, so if they don’t use Pinnacle, they clearly don’t know much about sports betting.

A clever illustration of survivorship bias

There are many relevant examples of survivorship bias, but a particularly clever demonstration was performed by the famous English illusionist Derren Brown in a 2008 program titled "The System," which illustrates how misleading it can be.

The show was based on the idea that a system could be developed to "guarantee a winner" in horse races—a claim regular bettors are familiar with. The show followed Khadisha, to whom Brown had anonymously sent five consecutive correct horse race predictions. There was no trickery involved; the predictions were fair and accurate, and the program built towards a climax with a sixth and final bet where, with confidence boosted by Brown's 100% success rate, Khadisha invested $4,000 of her own money... and lost.

Of course, there was no system; Khadisha was simply the product of survivorship bias.

In reality, Brown had started by contacting 7,776 people (a sufficient sample size) and divided them into six groups, giving each group a different horse for a six-horse race. Note that the number of variables is just as important as the number of predictions in determining how quickly the initial sample size shrinks.

After each race, 5/6 of the people had lost and were eliminated from the system (like our failing clowns), and an equal proportion of the survivors were randomly sent another selection. Khadisha happened to be the ultimate survivor, winning five times in a row.

The fundamental lesson for sports betting is that anyone can have a lucky streak, and the more improbable something is, the bigger the role luck plays. If your typing clown produces the Complete Works of Shakespeare from a sample of several billion, don’t get too excited. However, if he repeats the feat, take a closer look.

A simple formula to evaluate a tipster's abilities

A simple way to evaluate a tipster's true abilities is to take the square root of the total number of selections and add that number to half of the total plays made:

For example, if a tipster has 400 predictions, the square root would be 20, which, when added to half of 400, gives a total of 220 theoretical wins.

If the tipster is 20 selections above 200, they are two standard deviations above average. There is about a 1 in 40 chance of a 50% bettor achieving this, or only a 2.5% chance of it happening. So, a player with 400 selections would need to score 220-180, or 60-40 with 100 selections, to be this rare.

Without being a master statistician, you can quickly see that the more selections you can review, the easier it is to evaluate a player. In many cases, it is safer to follow someone with a lower winning percentage if they have made many more bets.