Monday, March 15, 2010

Smash that Crystal Ball

It's that time of year again.

Yep, the time of year when the major Bay Street and Wall Street firms, along with the major mutual fund companies, parade their gaggle of economists and strategists in front of every camera, microphone and scribbling print journalist in order to emboss their firm's logo on the impressionable minds of wealthy Canadian investors. Eventually, the thinking goes, your current Advisor will have a poor year, or a poor twenty years, and you will inevitably start thinking about moving your hard-earned capital to another, more prospective wealth management group. If their firm has caught your attention over the years more often than other firms, the thinking goes, you are more likely to seek them out than their competition. 

This is one of the more common ways in which firms compete for your business.

They know from experience that you won't remember what their so-called 'expert' proclaimed on the news last year, or last month, about the shape of things to come. They know if doesn't matter what they say so much as the fact that they are out there, in the media, saying anything.

But there is another, more insidious reason why each major firm has a variety of experts on staff loudly proclaiming their views year after year. The reason is simple: banks, mutual fund companies and investment firms make no money while clients are sitting in cash. Each time the investing public observes an expert loudly proclaiming that gold is going up, or interest rates are going up, or Canadian banks are going up, a few of them take heed, call their Advisor, and make changes to their portfolio. And each time they make a change, the Advisor, investment firm, or mutual fund company makes money.

Unfortunately, on average the investing public doesn't. In fact, by listening to these quacks the average investor earned 8% per year less than the stock market from 1986 through 2009. (Source: Dalbar, 2009)

That's why, at this time of year it is especially important to remind you of the abysmal track record these 'experts' have had over the years.

But before presenting the ugly details, I want to emphasize that investors should not feel disheartened by the evidence that financial marketing and media is dominated by loud, overconfident shills and mountebanks. On the contrary, investors should feel liberated to pursue other interests rather than reading or watching business news. For those that enjoy the cognitive 'sport' of investing from the standpoint of strategy and game theory, feel free to explore the latest economic, financial, ideological or philosophical fads with your colleagues and friends as provocative dinner conversation. This type of thinking keeps the mind young, after all.

Just don't orient your portfolio on the basis of your conclusions, or the conclusions of other thinkers. We are all bound to be wrong far more often than we are right. For that is the nature of complex, dynamic systems like the markets.

Now, here is the evidence. (Or, if you are pressed for time, click here to skip to the sizzle at the end). Note that these charts are sourced from James Montier's book Behavioural Investing (2007):

Chart 1. Consensus bond yields forecasts 1 year out vs. actual

Chart 2. Consensus S&P500 level 1 year forecasts vs. actual

Chart 3. Consensus S&P500 aggregate earnings 1 year forecasts vs. actual

Note that in all cases, strategists, analysts and economists do an excellent job of describing what happened or is currently happening; that is they do an excellent job of observing the obvious. Unfortunately, they demonstrate no predictive ability whatsoever, as their forecasts of likely outcomes one year out have no meaningful value whatsoever.

Source: Despair.com

Still not convinced? The following chart shows the percentage error of analyst earnings forecasts from 24  months prior to an earnings announcement through to the date of the announcement, using data from 1986 - 2000. Not surprisingly, analysts demonstrate significant over-optimism in their earnings forecasts from two years out, while their forecasts narrow toward the actual number by around 2 months prior to earnings. The average error at 1 year is approximately 10%, and by a month prior they are slightly pessimistic. Of course this slight pessimism then allows the companies they cover to beat estimates slightly, which often results in a price jump.

Chart 4. The walk down to beatable earnings.
Source: Dresdner Kleinwort Wasserstein Macro Research

The error rate would not be so worrisome if it weren't for the high level of confidence that investment professionals imbue on their predictions. This effect is perhaps best illustrated using the results of a study by Torngern and Montgomery (2004). The study set laypeople (psychology undergraduates, the perennial guinea pigs) against investment professionals in a competition to select the stock that they thought would outperform over the next month from pairs of stocks. All the stocks were well known companies, but participants were given information such as the industry and prior 12-month performance for each stock as well. Participants were asked to choose the best performer from the pair, and to provide their level of confidence in their choice.

Over many picks, one might hope that when participants were 50% confident that their choice was right, they were accurate about half the time, and when they were 90% confident, they were right almost all the time. In fact, as you can see from the chart below, a person's confidence level was largely irrelevant to their accuracy over time. In other words, having greater confidence in a choice did not lead to higher accuracy levels. In fact, at extreme levels of confidence (>80%), professionals were actually less likely to get it right. At a 90% level of confidence, professional investors actually got it right only 15% of the time, while at a 55% - 75% level of confidence they achieved about 40% accuracy.

Chart 4: Accuracy and confidence on a stock selection task
Source: Torngren and Montgomery (2004)

It is important to remember that over a 1-month time horizon the results of these stock choices are almost random, so we are not out to skewer professionals on the basis of their accuracy in this test. Instead, we are left to wonder why anyone should have expressed such high levels of confidence in their choices. When asked this question, the layperson group admitted that they were mostly guessing, but also placed some emphasis on the previous month's returns (ahh, momentum at work). In contrast, almost no professionals admitted to guessing; instead, they attributed their choices to 'Other knowledge' about the stocks, and 'Intuition'. Incidentally, the only factor with any predictive power in this example, however small, is the previous month's results (momentum).

Chart 5. Average rating of decision input importance
Source: Torngren and Montgomery (2004)


The quantum leap in thinking that I want to convey with this post is that there is indisputable empirical evidence that the world is too complex to enable accurate forecasting. Axiomatically, people should consider expert forecasts as no more than entertaining narratives - brain candy to stimulate the imagination. Even complex mathematical models are relatively poor predictors of the future beyond a certain time threshold. The best we can hope for is an assessment that an existing dynamic or trend is likely to stay on a certain course, or alternatively that the course is changing. Forecasting the direction or the magnitude of the change in trend is empirically impossible.

So what can you do?

Clients know that this is the basis of our investment approach, and that our approach is entirely based on actual historical evidence, not expert forecasts (or our own):

  1. If something is trending it is likely to continue trending
  2. If something is trending most strongly, it will likely trend most strongly over the following several months
  3. When the direction or strength of the trend changes, adapt.
The results from a strategy which follows these simple rules speak for themselves.

Source: Acorn (2010), Butler|Philbrick & Associates
Results are pro forma and for illustrative purposes only.

0 comments:

Post a Comment