Predicting Markets, or Marketing Predictions

Mark Twain suggested that it is better to remain quiet and be thought a fool, than to open your mouth and remove all doubt.

Would that the 'gurus' who populate the investment and economics landscape would heed Twain's advice. Of course, that will never happen.

We know from studies of expert judgement that gurus who make nuanced predictions and hedge their bets attract much less attention than experts who spin dramatic predictions with unswerving confidence. As a result, firms are predisposed to encourage gurus to voice strong opinions and divergent views that stand out from the crowd. Unfortunately, the qualities that make some gurus more marketable than others are likely to render them less accurate: balanced experts tend to be more accurate than loud ideologues, and their opinions tend to be less damaging when they go wrong. 

And even the best experts get it wrong a lot. In fact, they get it wrong more than they get it right. How do we know?

The best and most comprehensive study of expert judgment was performed by Philip Tetlock. In 1985 Tetlock, fascinated by his previous experience serving on political intelligence committees in the early 1980s, set out to discover just how accurate expert forecasters were in their predictions of future events. Over a span of almost 20 years, he interviewed 284 experts about their level of confidence that a certain outcome would come to pass. Forecasts were solicited across a wide variety of domains, including economics, politics, climate, military strategy, financial markets, legal opinions, and other complex domains with uncertain outcomes. In all, Tetlock accumulated an astounding 82,000 forecasts.

This represents an incredible body of evidence about expert judgment, and Tetlock's analysis rendered several astounding conclusions:

• Expert forecasts were less well calibrated than one would expect from random guesses
• Aggregated forecasts were better than any individual forecasts, but were still worse than random guesses
• Experts who appeared in the media most regularly were the least accurate
• Experts with the most extreme views were also the least accurate
• Experts exhibited higher forecast calibration outside of their field of expertise
• Among all 284 experts, not one demonstrated forecast accuracy beyond random guesses

In short, experts would have delivered better forecasts by flipping coins. But there was a silver lining.

Tetlock also tracked some simple, rules based statistical models alongside the experts to see if these models would be competitive in terms of forecast calibration. He found that many simple models performed with substantially better calibration than the experts, and delivered accuracy well beyond random chance. Chock another one up for the quants.

You might be wondering whether there are any similar types of studies conducted specifically in the area of financial markets. You're in luck, as there are have been several.

CXO Advisory has been tracking and publishing gurus' forecasts of market direction since 1998. Recently, CXO published a review of all 6,459 forecasts from all of the market 'gurus' that they tracked from 1998 - 2012. Specifically, the gurus were graded on their ability to call the direction of the market, but were not penalized for missing the magnitude of the move.

Over 14 years, CXO concluded that the average guru's accuracy in calling the direction of the market has been about 47%, or slightly worse than a coin toss. The following chart shows how the accuracy of forecasts has stabilized over time around the 47% mark as the sample size expanded over time. In other words, the experts were less reliable than flipping coins.

Source: CXO Advisory

The evidence does not end there. The following charts, sourced from James Montier's incredibly useful book,  Behavioural Investing (2007), show aggregate forecasts from Wall Street's most famous oracles through time, next to the actual trajectory of the forecast variable. 

Chart 1. Consensus bond yields forecasts 1 year out vs. actual

Chart 2. Consensus S&P500 level 1 year forecasts vs. actual

Chart 3. Consensus S&P500 aggregate earnings 1 year forecasts vs. actual

In all cases the analysts appear to do a noteworthy job of describing what just happened, but appear to have no vision whatsoever about what is about to happen next. This applies to interest rates, the level of stock indices, and aggregate earnings.

Source: Despair.com

Do any experts get it right? What about the experts at the Federal Reserve who are in charge of setting interest rates? Can they predict the magnitude or direction of interest rates just six months hence?

A working paper entitled "History of the Forecasters: An Assessment of the Semi-Annual U.S. Treasury Bond Yield Forecast Survey" (Brooks & Gray, 2003) studied the ability of Federal Reserve economists, including Alan Greenspan, from 1982 - 2002 to discover whether the group of experts that sets interest rates is able to effectively forecast their trajectory through time. 

Chart 3.

Source: (Brooks & Gray, 2003)

Again we see a strong talent for describing what has just happened, but no talent whatsoever for predicting what will happen next. Just how poor was the forecasting ability of Fed economists, including sitting Fed Chairman Alan Greenspan, over the 20 year survey?

Chart 4. 

Source: (Brooks & Gray, 2003)

The scatter plot above shows how Fed forecasts of interest rates just six months out are negatively correlated with actual outcomes. The r-squared of the regression is 7%, which is not statistically significant, so don't bet the farm against the Fed either. The point is, they can't forecast any better than anyone else.

There is ample evidence that strategists and gurus are unlikely to add much value to the investing process - at least where the goal is to grow your portfolio. Our next article will address another ubiquitous observation in wealth management - overconfidence - and discuss solutions for disillusioned investors looking for a new direction with better odds of success.

Track Records are Rubbish (or Why Managers are Factors in Drag)

As usual, you are the butt of the joke.

Everywhere you turn, you are bombarded with 1, 3, and 5 year track records for investment products. The investment management industry knows that you are influenced by percent symbols preceded by large numbers, so they market products with the best 1, 3 and 5 year track records, prominently featuring them in newspaper and TV advertisements, knowing that you will be unable to resist the urge to chase into those funds to avoid missing another year of riches. 

Source: fanpop.com

Unfortunately, as you probably surmised, this almost never works out. The products with the best track records over the past few years are not the products that deliver the best track records in subsequent years. In fact, there is strong evidence that chasing managers with strong 3 and 5 year track records is actually harmful to your portfolio health. This article will explore the evidence against using a 3 to 5 year track record to make investment decisions, and point to some alternative solutions.

3 Year Track Records Are for Suckers

The most commonly cited study of actual retail investor behaviour is the Dalbar Quantitative Analysis of Investor Behaviour which summarizes findings about mutual fund investor behaviour over the past 20 years. We plucked two important, and we think related, facts from the piece and summarized them in charts 1 and 2 below. Chart 1. shows the average holding period for each class of mutual fund (stocks, bonds and balanced funds), and Chart 2. shows the realized returns to actual investors on their stock and bond mutual fund holdings, over the period 1991 - 2011. 

Chart 1. Stock and bond investors hold funds for 3 years; balanced investors give it one more year

Source: Dalbar, 2012

Chart 2. Actual results to mutual fund investors vs. stock and bond benchmarks - 1991 to 2011

Source: Dalbar

From Chart 1. we can see that, on average, retail investors go chasing into new 'hot funds' about every 3 years or so; balanced investors seem to hang on a little longer, for reasons we explored in this article. Chart 2. highlights the insanity of this approach. Equity investors earned 4% per year less than the large cap stock benchmark over the same period, while bond investors fared even worse. Sure, about half of the decay can be attributed to fees and taxes, and a portion to bad market timing (strangely we didn't receive any calls from investors in a panic to buy in early 2009), but a measurable portion of the lag is due to misguided product selection on the basis of three year track records.

Chart 3., from the annual SPIVA report on active manager performance (2011 version) demonstrates the tendency for managers who demonstrate top quartile performance over a 5 year period to fall out of the top quartile over the next 5 years. One might randomly expect 25% of the managers who rank in the top quartile in the first 5 years to again rank in the top quartile during the second period. In fact, just 6% of these managers actually persist in the top quartile, implying a very substantial reversion to the mean effect.

Chart 3. Mutual fund manager performance persistence over 5 year periods

Source: SPIVA (2012)

Chart 4. shows that institutions act largely on the same schedule as retail investors, with manager termination decisions based largely on 3 to 5 year trailing results. This is not surprising because institutions are run by humans too. 

Chart 4. Average evaluation period for manager termination by pension funds

Source: Employee Benefit Research Institute

Chart 5. clearly illustrates the impact of this phenomenon in the pension space. The grey bars represent the average annualized performance of terminated managers in the three years prior to, and three years subsequent to, their termination. The white bars represent the performance of replacement managers in the same years. Clearly institutions are hiring managers with exceptional historical track records over trailing 3 year periods, and firing managers with poor track records. The joke is on the institutions, however, since on average the fired managers go on to outperform the hired managers over the subsequent 1, 2, and 3 year periods!

Chart 5. Excess returns to terminated and newly hired managers in the 3 years prior to, and subsequent to, termination

What an incredibly frustrating reality for retail and institutional investors alike. How can it be that managers with the best track records over as long as 5 years don't work out to be great investments in subsequent years? Does a track record mean anything? For the most part, we don't think so.

Manager Skill, or Factors in Drag?

It is important to remember that most investment managers are human beings. (I say 'most' because a minuscule but growing portion of managers are actually computers). As such, most managers are really just overconfident, incoherent collections of habits, assumptions, ideologies, and cognitive and emotional biases emanating from the most dangerous black-box of all - the human mind. Sometimes these habits, assumptions and biases are aligned with the market, and the manager does well. Sometimes the manager is 'out of sync', and he does poorly.

In reality, it is better to think of managers as inconsistent conduits to somewhat persistent factors that manifest in markets from time to time to drive outperformance from a certain investment approach. In the equity space, academics suggest that 4 factors explain the majority of long-term stockpicking performance. One might infer that the success or failure of most managers over any period of 3 to 5 years largely depends on whether the manager's biases were aligned with one of the following factors that happened to also be working over the same period.

• Market factor: this is commonly referred to as beta, and refers to the fact that most stocks tend to move in the same direction as the index
• Small-cap factor: it has long been recognized that small-cap stocks outperform large-cap stocks over the long-term
• Value factor: cheap stocks tend to outperform expensive stocks over the long term
• Momentum factor: stocks that have gone up over the past 1 to 12 months tend to outperform over the long term

Recently, academics and practitioners have reluctantly added a low volatility (or low beta) factor to explain the observed outperformance of low volatility stock portfolios, and we will include this factor in our analyses below.

Chart 6. offers graphical evidence of the value and momentum effect within large-capitalization stocks. Specifically, the chart demonstrates how a strategy of rotating into the strongest stocks every month, and a strategy of rotating into the cheapest stocks every year, have outperformed a strategy of holding the broad stock market index since 1927.

Chart 6. U.S. large capitalization stock market value and momentum factor tilt portfolios

Source: Ken French database

It is our assertion that clients should be much less concerned with the track records of individual managers, and much more concerned with the performance of a manager's style factor. If you can identify what factors (or what mix of factors) is most likely to generate the strongest risk-adjusted returns over your rebalance horizon, this information is of much greater value than identifying which managers might outperform their style benchmarks.

This assertion is strongly confirmed by results from Financial Product Differentiation Over the State Space in the Mutual Fund Industry by Li and Qiu (2010), who showed that 95.7% of cross sectional mutual fund performance is explained by the traditional four Carhart factors: market beta, small-cap, value, and momentum.

If 96% of mutual fund performance is explained by style factors, and most managers underperform the market and their style benchmarks, then investors should concentrate on allocating to factors, not managers. In the real world, when the value factor is outperforming, value biased managers will be high-fiving each other at the water cooler and celebrating their unique and robust stock-picking talent. Growth (or anti-value) managers, meanwhile, will be crying in their beer and lamenting the 'broken' market that isn't cooperating with their investment bias. Chart 7. clearly illustrates this effect, as value managers outperformed for the first four years of the last decade, gaining 26% vs. their growth oriented peers by late 2006.

Chart 7. Value vs. Growth: Luck or Skill?

Source: Stockcharts.com

However, just as value managers were buying their Porsches and brownstones in 2006, their performance streak ended. Over the next 6 years, value managers endured a steady diet of crow while their growth brethren feasted on rich returns. Chart 8. illustrates the same phenomenon for managers with small vs. large-cap biases over a 20 year horizon, but the same effect plays out in all factors: dividend stocks, momentum stocks, low volatility stocks, etc.

Chart 8. Small cap vs. Large Cap: Luck or Skill?

Source: Stockcharts.com

If style factors explain 96% of equity manager performance, and a single factor  can deliver 25% - 50% or more of outperformance over a four year period, you can see that the decision to allocate to either value or growth, small-cap or large-cap (insert any factor here) completely dominates the decision about which specific value, growth, small-cap or big-cap manager to use.

Surely Buffet is a Special Case, Right? Wrong.

Interestingly, a group at Yale investigated the full trading history of Warren Buffet's investment vehicle, Berkshire Hathaway to discover whether and which systematic factor exposures can effectively explain the seeming miraculous long-term outperformance Buffett has delivered over the years. The authors regressed the monthly returns to Buffet's portfolio (assembled through 13F filings) against the same 5 factors plus another factor, 'Quality', as defined in Asness, Frazzini, and Pedersen (2012b). They discovered that Buffet's strategy was essentially to purchase low beta, high quality stocks (using the long held academic definitions), and lever his portfolio by 60%. The authors regressed Buffet's returns against their 6 factors and then simulated the growth of the equivalent factor portfolio (green line) over Buffet's full investment horizon and compared it to Buffet's actual performance (blue line). The results are in chart 9. below.

Chart 9. 

Regarding the ability of factors to explain away Buffet's magical performance streak, the Yale authors suggest:

We see that Berkshire loads significantly on the [low beta] and [quality] factors, reflecting that Buffett likes to buy safe, high-quality stocks. Controlling for these factors drives the alpha of Berkshire’s public stock portfolio down to a statistically insignificant annualized 0.1%, meaning that these factors almost completely explain the performance of Buffett’s public portfolio.

Style Boxes Are Silly

The small-cap and value factors form an integral framework for many institutions, as they are the two dimensions used in a traditional 'style box' model. Investors who are guided by this model seek to gain exposure to each 'style box' by seeking out top managers in each style.

Figure 1. Morningstar's Style Box

Source: Morningstar

In reality, this is just a misguided method of factor investing; misguided because investors just end up with what is essentially a 'market' portfolio in the end, as they own both small and big stocks, and value and growth stocks. Further, 'large cap' and 'growth' are anti-factors, which means they have tended to deliver returns below the market portfolio over the long-term. For this reason (and others that are more nuanced), the style box model should be abandoned in favour of a model that embraces true systematic factor tilts against an efficient 'market portfolio'.

Should We Buy Strong Factors?

We've learned that we should be thinking about allocating to factors instead of managers because managers are just inconsistent factors in drag with a strong propensity to underperform. But how should we think about allocating to factors?

We tested a strategy of rotating into the strongest and weakest factors quarterly based on trailing 1, 3, and 5 year performance to see if any of these approaches work any better than holding an equal-weight portfolio of factor tilts, rebalanced quarterly.

Table 1. Summary of factor rotation strategies

Source: Ken French database, Standard & Poor's, Yahoo finance

We've highlighted the worst performing strategies in red. You can see that a strategy of allocating to the factor that performed the best over the past 3 and 5 year periods delivered the worst absolute and risk-adjusted performance over our sample period (1995 - 2012). In fact, the best approaches would have been to allocate to factors that delivered the worst performance over either the past 1 or 5 years, or to allocate equally to the best and worst factors over the prior 1 year period. A strategy of allocating equally to all factors and rebalancing quarterly delivered similar risk-adjusted returns, but lower absolute returns.

From the table above, we would conclude that there is a weak mean-reversion effect with equity market factors over 1 year and 5 years, and an even weaker momentum effect over a 1 year horizon. Clearly the worst strategies are the ones that are embraced by most investors: buying the strongest performing approach over the past 3 to 5 year period.

Future Directions

From a statistical standpoint, factors don't appear to exhibit either a meaningful momentum or mean reversion signal, so investors really don't have much hope of excess returns from chasing into managers with great track records or backing up the truck on managers with awful track records. 

However, both of these dynamics really come down to estimating which factor(s) will deliver the highest returns. If we take the perspective that simple performance offers no meaningful information about future returns, then we are left with optimization alternatives related to relative volatility, such as risk parity, or the covariance matrix, such as minimum variance. We explore these approaches at length in our paper, 'Portfolio Optimization with Factor Tilts', but here is a sneak peak. 

Chart 10. 5 Equity Factors, Minimum Variance, Rebalanced Monthly, 25% Filter, Portfolio Target Volatility (1%), Max 100% Exposure

Source: Ken French database, Standard & Poor’s, Yahoo Finance

Chart 11. 5 Equity Factors, Minimum Variance, Rebalanced Monthly, 25% Filter, Portfolio Target Volatility (1% daily), Max 200% Exposure

Source: Ken French database, Standard & Poor’s, Yahoo Finance

Mutual fund companies and institutional consultants will continue to feed your biases by advertising their best track records, but you don't have to fall for it. With a little research - and an open mind - you can uncover novel methods based on academically validated principles with a proven history of delivering market-beating returns with lower risk.

In our opinion, the most interesting and prospective extension of the concepts discussed above are in the form of Tactical Alpha rather than traditional security selection. This type of approach deals with the allocation of factors across multiple asset classes (see here from AQR[registration required] and here for our own paper), which allows for many more sources of return and diversification, which gets us much closer to investment Nirvana.

The Full Montier: Absolute vs. Relative Value

James Montier, one of our favourite thinkers and fellow evangelist for behavioural economics and evidence based investing, published a great report last year entitled, "I Want to Break Free, or, Strategic Asset Allocation ≠ Static Asset Allocation". The piece is a tour de force against the ubiquitous concept of "Strategic Asset Allocation" (SAA) which argues that investors should set an asset allocation at the start of their investment horizon that theoretically balances individual risk tolerance against long-term average market risks, and then regularly rebalance back to this target allocation.

As we have argued many times in this blog, traditional SAA is hamstrung because there is no mechanism to alter the asset allocation given changes in relative expected returns and risks from each asset class over time. Inconceivably, adherents to this approach would have advocated exactly the same mix of stocks and bonds in December 1999, when stocks were as overvalued as they had ever been in history, as they would have advocated in 1982 when stocks were dirt cheap. We have argued, and Montier apparently concurs, that investors should be constantly aware of whether prospective returns to stocks are high or low, and alter asset allocations accordingly through time.

Montier is a die-hard value investor, so he views prospective returns through this lens. He asserts that investors should only invest in markets when they offer the prospect of reasonable future real returns on the basis of reasonable valuations, and they should avoid markets when they offer low returns. While this seems eminently reasonable, in practice this logic can be very difficult for investors to adhere to because markets can continue to get more and more expensive (or cheaper) for many years, and investors often find it difficult to stand on the sidelines and watch the market shoot into the stratosphere. For example, markets entered a period of extreme valuation by many measures in 1994 and continued to push higher for 6 more years before finally succumbing to valuation levels that were, by some measures, more than twice as high as any other period in history.

For this reason, most wealth managers, institutions, and advisors practice Strategic Asset Allocation, which keeps investors fully invested in their target mix of stocks and bonds at all times. Where active bets are taken in pursuit of better performance, they take the form of relative value approaches where individual securities are selected for portfolios because they have lower valuations than their peers. This approach is starkly different than the absolute value approach described in the previous paragraph because with the absolute value approach, an investor will hold cash when no investments offer strong absolute returns, whereas relative value investors will always be fully invested, even when all investments are expensive and offer low returns.

Absolute Value: A Few Tests

We thought it would be fun to test an absolute value approach per Montier's prescription using our favorite long-term stock-market data source, Shiller's database. The following charts illustrate how an absolute return investor might have fared had he chosen to move out of stocks and into cash where the real absolute valuation of the stock market ended the prior month below a certain threshold. For ease, we chose the cyclically adjusted earnings yield as the valuation metric, which is just the reciprocal of the Shiller PE. We then adjusted the yield value for the realized year-over-year inflation rate to find the real earnings yield. Finally, we used an 'expanding window' approach to find the percentile rank of the real earnings yield to eliminate as much lookahead bias as possible.

Note that because we are using real earnings yield rather than nominal earnings yield, markets can get cheap or expensive in three ways:

• changes in inflation
• changes in earnings
• changes in price

As a result, while markets would appear to be quite expensive today based on nominal earnings yield, which is in the top quintile of all values over the past 140 years, the real earnings yield is less extreme because yoy inflation is so low. Should inflation pick up, the real earnings yield will decline, all else equal, which would alter the current state of our model (in stocks or in cash). 

Chart 1. Valuation based asset allocation: own S&P500 when valuation < long-term average, otherwise hold cash

Source: Shiller (2012), Federal Reserve

Using the expanding window approach, an investor would have spent about 80% of months in cash over the past 80 years or so, implying a fairly consistent expansion in PE ratios over time (we identify periods where market valuations are above our threshold with grey areas on the chart). Even so, by investing in markets only when they are truly cheap (> median real earnings yield) and holding cash otherwise, investors would have generated about 70% of the total return to stocks with less than half the volatility and 73% lower drawdowns since 1934. The reduction in drawdowns is especially important in the context of Montier's definition of risk: the permanent impairment of capital.

We know from history however that when markets move from being inexpensive up through fair value, they usually move well into the range of 'overvalued' before peaking. If that's the case, investors might choose to hold onto stocks until they move further into the expensive range. Chart 2. shows the performance of a strategy that holds stocks unless they are in the top quintile of valuations (bottom quintile of real earnings yields), and holds cash otherwise.

Chart 2. Valuation based asset allocation: own S&P500 when valuation < 80th percentile, otherwise hold cash

Source: Shiller (2012), Federal Reserve

By tolerating higher, but not extreme, valuations investors would have achieved higher returns than the markets overall (11.4% vs. 10.4%) with 26% less volatility and 40% lower drawdowns. As a result, the Sharpe ratio of this approach was 0.8 vs. 0.51 for buy & hold.

A more coherent approach to this framework might eliminate the absolute threshold entirely in favour of a scaled approach. For example, an investor might hold stocks in proportion to the percentile rank of the real earnings yield, and hold the balance in cash. For example when the real earnings yield is at the 80th percentile, the investor would hold 80% stocks and the balance of 20% in cash. Conversely, when stock earnings yields are at the 20th percentile, he would own 20% stocks and 80% cash. Again, this measure is based on valuations at the prior month's close, and using an expanding window to avoid lookahead bias. Chart 3. shows the results of this approach.

Chart 3. Valuation based asset allocation: own S&P500 in proportion to the percentile value of real earnings yield (hold balance in cash)

Source: Shiller (2012), Federal Reserve

This coherent approach avoids having to choose a cutoff threshold, which makes it more robust. Further, the Sharpe ratio is almost as high as for the model with an 80th percentile threshold, though in this case it is because of very low average volatility (5.1% vs. 9.6%), as the returns are considerably lower.

Lastly, we wondered how the same approach would have worked if instead of allocating to cash, we allocated to Treasuries. Chart 4. gives us the answer.

Chart 4. Valuation based asset allocation: own S&P500 in proportion to percentile value of real earnings yield (hold balance in Treasuries).

Source: Shiller (2012), Federal Reserve

Future Directions

James Montier is incensed by the ubiquitous calibration of strategic asset allocation with 'static' asset allocation because static allocation makes no accommodation for the fact that market valuations and commensurate expected returns fluctuate dramatically over time. Does it make sense for investors to be mostly indifferent to expected returns in setting their asset allocation targets?

In keeping with Montier's absolute value philosophy, we investigated several dynamic allocation strategies based on reducing or eliminating exposure to markets as they get more or less expensive, using the real earnings yield as our yardstick. In every case we tested the absolute value based approach delivered a higher Sharpe ratio, and a much higher ratio of returns to our approximation of Montier's measure of risk - maximum drawdowns.

The primary challenge with these approaches is that they require adherents to exit markets just as the party gets going into major peaks. Major market peaks manifest in the midst of extreme optimism and enthusiasm about the future which sweeps every last investor dollar into stocks over several euphoric years. It takes extreme fortitude to stand aside in cash during these long periods of parabolic market gains, and we know from the published studies in behavioural finance that most investors will buckle and go all-in just before the market peaks and rolls over.

Investment managers that adhere to this type of approach will suffer enormous career risks during these periods, and lose a large portion of their assets under management. Montier's current firm, GMO, admits to losing over 40% (!!) of total client assets in the several years leading up to the 2000 market peak, and John Hussman has experienced similar investor attrition over the last few years as his valuation models have kept him largely on the sidelines during the market's current bull market run. To paraphrase Keynes 'The markets can stay irrational longer than you can keep clients.'

We believe that successful investment management involves two key elements: 

1. Systematically apply an evidence based approach to investments that maximizes return with minimal risk
2. All else equal, and with a clear understanding of investors' behavioural flaws, apply a strategy with the highest probability that investors will stick with it over time
• avoid large drawdowns to reduce emotional decision making based on fear
• participate in major bull markets to reduce emotional decision making based on greed

There are several approaches that embrace these two broad qualities, but our preference is toward strategies that leverage momentum and volatility management. These techniques are practiced with demonstrable results by the best trend following managers, but we'd like to think our Adaptive Asset Allocation program represents another attractive alternative.