Investment psychology
Pseudo statistics
One of the greatest failures of investors revolves around faulty inductive reasoning. This is where we jump to conclusions from scanty or no evidence or just don’t think straight. Or, just as bad, we accept unsupported opinions of others
In my last post I discussed the Law of Large Numbers. In this post I will discuss the Law of Small Numbers.
Investors are bombarded daily with the opinions of analysts, advisors, pundits, journalists and bloggers purporting to be based on conclusions drawn from statistical analysis. Typical would be an opinion that the stock market is likely to advance next year. The reason given might be the assertion that when the S&P 500 has advanced more than 20% in a year, it is likely to advance the following year. The proffered evidence might be that, historically, the stock market has advanced 20% or more 17 times and that 14 of those occasions were followed by an advance the following year. What credence can we give to this kind of prediction? Investors are not trained statisticians. How can we sort the wheat from the chaff?
Let me say quite bluntly that there is absolutely no statistical basis for stock market advice from the previous paragraph. It’s simply an example of someone (uninformed pundit or influencer) jumping to a conclusion based on no or scanty evidence.
Even the experts get it wrong
Two statisticians, Howard Wainer and Harris Zwerling wrote an essay about an investment of approximately $1.7 billion made by the Gates Foundation which was intended to implement the findings of a study showing the characteristics of the most successful schools.
As described by Daniel Kahneman, referencing the Wainer/Zwerling essay: “One of the conclusions of this research is that the most successful schools, on average, are small. In a survey of 1,662 schools in Pennsylvania for instance, 6 of the top 50 were small, which is an overrepresentation by a factor of 4.
These data encouraged the Gates Foundation to make a substantial investment in the creation of small schools, sometimes by splitting large schools into smaller units. At least half a dozen other prominent institutions, such as the Annenberg Foundation and the Pew Charitable Trust, joined the effort as did the U.S. Department of Education’s Smaller Learning Communities Program.” (Kahneman, Thinking, Fast and Slow. 2011) p117
It is easy to construct a supporting narrative or story. One can easily imagine smaller schools giving more personal attention to students.
The problem is that the statistical conclusion is mistaken. The data established no causal connection between smaller schools and better student outcomes. It seems that the data would just as easily have shown that bad schools also tended to be smaller. The small schools were simply more variable in their outcomes.
Kahneman explains the statistics problem caused by the different sizes of the schools. He asks us to imaging two very patient helpers taking red and white marbles from a large urn that contains half red marbles and half white marbles. “Jack draws 4 marbles on each trial, Jill draws 7. They both record each time they observe a homogeneous sample– all white or all red. If they go on long enough, Jack will observe such extreme outcomes more often than Jill – by a factor of 8 (the expected percentages are (12.5% and 1.56%).” The drawing of the marbles is purely random. There is no causation at work in the outcome. The fact that Jack sees more all red or all white marbles is a pure mathematical fact. The fact is that in statistics, “small samples yield extreme results more often than large samples do”. (Kahneman, 2011) p110 (Emphasis Added)
This seems innocuous enough. But it seems even highly trained researchers can get it wrong.
The Law of Small Numbers
Most readers will know that Kahneman was a psychologist who was awarded the Nobel prize in economics. Chapter 10 of his wonderful book Thinking Fast and Slow is titled ‘The Law of Small Numbers’. The chapter focuses on our very human tendency to overgeneralize and find causes. He calls this cognitive error, with a smile and nod to the statistical Law of Large Numbers, the Law of Small Numbers.
As Kahneman puts it: “The strong bias toward believing that small samples closely resemble the population from which they are drawn is also part of a larger story: We are prone to exaggerate the consistency and coherence of what we see. The exaggerated faith of researchers in what can be learned from a few observations is closely related to the halo effect, the sense we often get that we know and understand a person about whom we actually know very little. System 1 runs ahead of the facts in constructing a rich image on the basis of scraps of evidence.” (Kahneman, Thinking, Fast and Slow. 2011) p114
Both pundits and the investing public are simply human. We have serious shortcomings around how we form our beliefs. The problem is drawing conclusions from minimal evidence. In statistics, a field of mathematically based inductive reasoning, the weakness is exemplified by small samples.
What happens in these situations is that we humans make cognitive errors (example being overgeneralizing) and compound the problem with behavioral biases (example halo effect). What results is a failure of inductive reasoning.
The investment guru
Most investors are aware that the money manager or investor who has beaten their benchmarks the last year or two is considered by many to be a whiz. In reality the performance is just as likely due to luck i.e. to chance. Two years and even much more is not enough of a track record to draw any conclusions.
Look at the following three sequences. We track the investment performance of three investors. B is bad performance in the stock market – underperforming the average investor. G is good performance – outperforming the average investor performance. The first investor has a track record over five years of BBBGGG, that is, they had three bad years followed by three good years. Since our benchmark is the average performance of all investors, in any given year half will underperform and half will outperform. Our second investor had five good years!
BBBGGG
GGGGGG
BGBBGB
Are all the sequences equally likely? We can’t help looking at the second investor as being the guru. But, in truth, each sequence is equally likely. Intuitively we conclude the second investor is the guru and that their results are caused by their skill. We have made a cognitive error. We have made the error because our sample size is too small. We need many more years of outperformance before the Law of Large Numbers kicks in. We suffer from the all too human tendency to draw conclusions from a minimal amount of evidence. (Kahneman, 2011) p115
Emerging markets and small companies
It seems experts, who should know better, often fail to pay sufficient attention to sample size. Stock market analysts and stock market strategists make frequent use of statistics or, at least, purport to. One might question whether they have sufficient grounding in mathematics to know what they are doing. Let’s reflect on a professional stock strategist opining on the outperformance of emerging markets and the commonly mentioned outperformance of small companies. Emerging market stock markets have far fewer listed stocks than developed market exchanges. For this reason alone, their results would be more variable than a larger market. As well, what is the time frame. Is it long enough? Are other economic factors at work? As for company size performance we need to ask hard questions about sample sizes and whether the sample periods are biased in some fashion, or whether there is a survivor bias in the data, so as to make it dangerous to generalize.
These studies are usually accompanied by a narrative, a story, such as in the former case, that emerging economies contain greater opportunities for high growth, and in the latter case, that smaller company are nimbler and are growing from a smaller base.
Dogs of the Dow
There is an amusing investment theory called the Dogs of the Dow. It is thought that if you invest each January in a collection of the stocks in the Dow Jones Industrial Average that performed the worst the previous year you will do well. There is actually some conceivable statistical sense to the notion. It is based on regression to the mean. Unfortunately for this folk theory, the Dogs of the Dow is subject to the Law of Small Numbers.
We intuitively come up with the idea that the worst performers will do better because they will get their act together, or that management will be more motivated. We might even seize on regression to the mean. In fact, any regression to the mean would be a purely mathematical fact and we don’t have a clue whether our sample is large enough to have any statistical meaning. And there might be other factors at play.
Another similar one is Sell in May and Go Away. This and similar folk ideas are frequently invitations to engage in market timing. This is decidedly a no no.
In all these situations, we intuitively look for patterns, scanty or even imagined evidence, causal connections and so on.
Charts and other patterns
Some investors and traders look for patterns such as areas of resistance, areas of support, basing formations, break outs, head and shoulders formations and so on. For the most part these are totally unreliable. Overbought and oversold indicators are a frail basis for making stock purchase and sale decisions. The shorter the time frame the less reliable they are as indicators of anything. They may be pattern illusions.
There are two principal reasons for this. Even purely random data series can show apparent patterns. It is hard to distinguish the illusion from the real. The second reason is that shorter term charts/data suffer from the Law of Small Numbers.
Other statistical problems
Here are a few others. There is the common problem around back testing to prove various quant strategies. A thousand different strategies can be tested. Difficulties arise when cherry picking of results occurs. Some of the thousand tested strategies are bound to show up well in a back test. That doesn’t mean they will work well in future. These back tests also suffer from the weakness of non-stationarity of data when the world is changing and what’s happened in the past is no guide to the future.
There is also the problem of survivorship bias in data sets. If 100 companies are looked at over a 20-year period and ten of the companies have gone bankrupt, it is wrong to use the remaining 90 alone to measure performance.
Another common problem is when a data series is thought to follow a normal distribution but in reality, follows a different distribution. For example, stock prices apparently tend to have longer tails than the normal distribution.
For balance, here’s the flip side of the Law of Small Numbers problem. A lot of people believe a crowd is wiser than the individuals in the crowd. The Wisdom of Crowds is actually purely a statistical phenomenon.
This is the phenomenon where you have a large group of people estimating some quantity. The average of their estimates will tend to be very close to the actual number.
The mistake a lot of people make is the narrative that when a lot of people put their heads together some collective wisdom comes out of it. So, for example, some people believe Wisdom of Crowds applies to the stock market. It doesn’t. See my post: Does Wisdom of Crowds apply to earnings estimates, price targets, value estimates and stock prices?
This is not to deny the value of collaboration in groups of people. But, that is another idea entirely.
Conclusion
The writer doesn’t have the expertise to answer all these questions. I am not a mathematician. I studied math many years ago but am not an expert today. I do know enough to ask them and be skeptical of so-called statistical conclusions. The opinions of analysts and pundits often offer pseudo statistical rationales to invest in a particular stock, asset class, type of stock, country or sector. The suggestion is that these recommendations will offer better returns in the future. The conclusions could easily be generalizations based on limited and inadequate data and pseudo statistics.
+++++++++++++++
Other posts on investment psychology
This post is part of a series. Readers are invited to read Investment psychology explainer for Mr. Market – introduction This will give you a better understanding of some of the terms and ideas and give you links to other posts in the series.
+++++++++++++++
You can reach me by email at rodney@investingmotherlode.com
I’m also on Twitter @rodneylksmith
+++++++++++++++
Want to dig deeper into the principles behind successful investing? Click on the ABOUT tab to read an introduction. It will help readers get the most out of the Nuggets of Investing Wisdom blog.
+++++++++++++++
Check out the Tags Index on the right side of the Home page that goes from ‘accounting goodwill’ to ‘wisdom of crowds’. This will give readers access to a host of useful topics.
+++++++++++++++
You can also use the word search feature on the right-hand side of this page to find references in both Nuggets blog posts and also in the Motherlode.
+++++++++++++++
To explore the Motherlode, click on the Motherlode tab
If you like this blog, tell your friends about it
And don’t hesitate to provide comments or share on Twitter and Facebook