Chapter 1

The Sure Thing Almost Nobody Plays

Imagine you are entering a deluxe, well-appointed casino. Off the lavish entry foyer, there are two ample gambling wings, one hued in reds, the other in muted greens. The red wing looks enticing, but if I may insist, let's first enter the less crowded green rooms to watch the action.

The atmosphere is unhurried, the blackjack tables are sparsely attended, and every player sits behind a mound of green and black chips. You think at first you've come to the wrong place. You see the ordinary table limits, the ordinary clothes, the ordinary games. But then how did these ordinary people get such piles of money?

Then it comes to you. They're all winning. In fact, as you walk around the green wing, you hardly can find a losing player. You know, of course, that the average house take on table games is 5%, but as you count winning and losing hands, you realize these players are getting a better break. They seem to be gaining at a rate of 60% to 40%. You start fresh and take another count. The results are the same.

A pit boss appears at your shoulder.

"Excuse me," you say, "but can this be right? The odds favor the players?"

"Yes, indeed. The odds in the green room usually run 60 to 40. It's been that way since we opened."

"But...most of the players must go away winners."

"They sure do. At those odds, we calculate that 9,999 out of 10,000 make money. At our high-stakes tables in the back, they do even better, with winners running about 20,000 to 1. It's a good thing we get so few players, or they'd break the house."

Somewhat amazed, you thank him and shake your head. There's no time to lose, you decide, but you'll need more than the few dollars you have in your pocket. You hatch a plan to gather your life savings, come back to the casino, and win the bundle you've been dreaming of.

On your way out, you glance into the red wing. The action level is much, much higher. The room is crowded and fairly roars with excitement. Can it be even better here, you wonder? Curious, you go in. Players bet multiple table positions, wave frantically for change, entreat the gods for luck. You see few green and black chips, fewer winning players. The piles of chips in front of them are dwindling with each hand. In fact, the odds are worse than normal. Again, you start to count. Although the players continue to excitedly toss in their chips, the odds appear to be maybe 60 to 40 in favor of the house. Once more, your curiosity whetted, you walk over to a pit boss and ask her the odds at these tables.

She tells you what you suspected. They are 60 to 40 in favor of the casino. Warming up to the subject, she chuckles and says, "This room coins gold for the casino, the chances are 9,999 in 10,000 rounds that we wind up winners." You don't have to be a genius to see that this is obviously not the place you want to be.

You go home and get your stash. You return to the casino with your fistful of money, excited, eager for action, all the time figuring how you'll do even better at the game. But then a strange thing happens. You walk into the red wing and start to play.

On Markets and Odds

Sounds like a pleasant dream rapidly degenerating into nightmare. Yet it happens daily in the market. Moreover, the odds in the green and red rooms are not just attention grabbers. They're very real. In the marketplace, in fact, some investments steadily make money, while others lose consistently at these identical odds. Like the casino, most people want investments that have a good chance of beating the market. Yet, despite considerable time and effort devoted to the mission, they gravitate toward investments where swarms of enthusiastic players are endangering their savings at odds similar to those in the red room.

It seems surrealistic, but it isn't. Generations of investors have passed up sure things to buy losing investments. We all know, for instance, that for safety we should invest in treasury bills and government bonds. The investment environment has changed radically in the postwar period, however, as we'll see in detail in chapter 13. Treasury bills and government bonds, gilt-edged securities for centuries, are now surefire ways to destroy your nest egg. Conversely, investments always viewed as more speculative, such as common stocks, have become outstanding vehicles to protect and enhance your capital.

Yes, all the prudent rules of saving we learned at our fathers' knees are out the window. Since the Second World War there has been a revolution every bit as violent to the old order of investing as the French Revolution was to the old order of Europe.

Revolution does not have to mean the destruction of your savings. As the Industrial Revolution shifted fortunes from the titled nobility to entrepreneurs, as the railroad revolution enriched the Vanderbilts and Goulds, as the information revolution enriched the Gateses and Jobses, so this investment revolution, if you know what causes the changing structure, can enrich you. In each of these revolutions, the odds strongly favored those who found the new road and disfavored those stuck in the same old path.

This book is about odds or probabilities in markets, and how to use them to your advantage. There are probabilities of success or failure in the marketplace as surely as in gambling, business, or warfare.

Napoleon was the greatest probability player of modern warfare. Most often outnumbered, he won by moving fast and concentrating his forces on the battlefield on the enemy's weak points. Yes, there was brilliance in his strategies, but a good part of his military genius lay in his knowledge of the odds in any situation. On the first Italian campaign, for instance, he gambled on the perilous dash through Genoese territory rather than throw his ragged, demoralized men against the fortified Alpine passes, and was able to defeat the numerically superior but divided Austrians by keeping his forces unified. While poised precariously before the gates of Vienna, however, he gave the Viennese a generous treaty rather than take a chance on being cut off from his supplies.

The story is told of Napoleon playing cards with his generals and staff to pass the time en route to Egypt with his invasion fleet. He heard one of his aides whispering in the background and asked him to speak up. The aide, fearing for his life, stammered, "I said, General, you are not playing fairly." Napoleon said, "That's true, but I always give the money back at the end of the game."

General Murat, who was to become one of his legendary Marshals, spoke up with trepidation, though he had always been fearless in battle. "General, if I may be so bold, can I ask a question?" Napoleon nodded and he continued, "If you give the money back, why then do you cheat?" Napoleon replied, "Because I want to leave nothing to chance."

"Interesting story," you might say, "but how does it apply to the marketplace?" Believe it or not, despite appearances, the market does not run on chance or luck. Like the battlefield, it runs on probabilities and odds. These control markets every bit as strongly as they do roulette, blackjack, craps, or any other game in a casino. Unlike Napoleon at cards, we don't have to cheat, though our survival does depend on staying ahead of the opposition. As this book will show you, the odds in the marketplace can be turned decisively in your favor.

What is behind the probabilities in markets? Is it a new system to divine the future, or differential equations critical to the development of physics and now used extensively in economics? Millions of dollars' worth of the best research money can buy? No. Our probabilities are not built on any of these techniques or on foreseeing the direction of markets, or on any other conventional approach. Rather they are embedded in investor behavior in the marketplace.

I know it sounds strange. "Psychology," some people might snicker, "is the last place in the world to find any sort of reliable odds." It's true all the same. Betting on well-defined patterns of investor behavior will give you higher probabilities -- strongly backed by statistics -- than any other method of investing in use today. These are the odds available in the green wing of my imaginary casino. But how do you arrive at them?

That is what this book will try to show you. But the last thing I want to do is ask you to take my advice or anybody else's on faith. Too many systems and faith healers already clutter the marketplace. None that I know of do what they claim -- beat the market. And most cost. We will carefully look at the odds of which investment methods work and which don't, forgetting the popularity of one technique or another.

To start, we'll look at the two methods heavily favored by knowledgeable investors: the use of professional money managers and the efficient market hypothesis. The first relies on the "edge" the professional investor can give you by banking on his or her knowledge, skill, and experience to turn the odds in your favor. Do these professionals understand the new market dynamics that I have discussed briefly? Do they win big in this dramatically changed environment? To find out, let's look at the expert record.

Great Expectations

Professional investors manage large pools of money for mutual funds, pension funds, bank trust departments, money management firms, insurance companies, and similar financial institutions. The assets under their management are growing phenomenally, surpassing $14.8 trillion by the end of 1997, and expanding faster than the GDP (gross domestic product). They control over 70% of the trading and 50% of the stock holdings on the New York Stock Exchange. Their buying power staggers the imagination. By 1997 they were estimated to be buying and selling $2 trillion of stocks annually, generating $10 billion of commissions in the process. They claim the crown as the elite of their profession.

The professional money manager is believed to bring a new depth of knowledge to securities markets. Armed with the best research money can buy, and legions of well-trained, battle-hardened colleagues, they were said to have turned the odds decisively in their favor. The change was welcomed by knowledgeable observers of the financial scene. They urged the uninformed and emotional small investor to realize that he was ill equipped to deal with the complex modern market and to turn his money over to one of these seasoned pros. Some writers on the subject have gone so far as to state that the average investor is "obsolete" today. So, with some knowledge of the role these professionals play in the nation's finances, we should now ask whether they have indeed brought the Age of Camelot to markets.

Unfortunately, in a word the answer is no. The great majority of money managers have consistently lagged behind the market averages. John Bogle, chairman of the Vanguard Group, one of the largest mutual fund organizations in the country, stated that 90% of money managers underperformed the market in every ten-year period since their performance began to be measured in the early sixties. Burton Malkiel, a professor of finance at Princeton and author of the widely read A Random Walk Down Wall Street, studied all domestic equity mutual funds and found that they underperformed their market benchmarks from 1971 to 1991. Information taken from Morningstar and Lipper Analytical Services, two of the largest services tracking the performance of mutual funds, measured the results for 3-, 5-, and 10-year periods to the third quarter of 1997 for the six major classes of stock funds. The results appear in Figure 1-1. As you can see, the market (as measured by the S&P 500) outperformed all categories of mutual funds for all periods (the one exception was small companies, which provided nominally better returns for five years). Numerous studies by pension-fund consultants, who monitor the results of hundreds, and in some cases thousands, of money managers, have come up with similar findings.

What conclusions can we draw from the tepid performance of professional investors? The rational and unemotional professional, according to financial lore, coolly gauges value and buys securities at bargain prices in periods of panic, when the public sells them with no regard to value. The portfolio manager, the legend continues, sells stocks in periods when the public is enamored with them, pushing prices to mania levels.

Evidence compiled by the Securities and Exchange Commission suggests otherwise. The much abused and supposedly emotional individual investor sold securities near the 1968 market top and bought at the market bottoms of both 1970 and 1974. The institutional investor, on the other hand, bought near market tops and sold at the bottoms. In the 1987 crash, the individual investor was scarcely involved. According to SEC records of the crash, almost all of the selling was done by professionals. The market panic in the third quarter of 1990, following the Iraqi invasion of Kuwait, demonstrated once again that professional, not individual, investors were the largest, and often most desperate, sellers.

The same pattern shows clearly with mutual funds. These funds should be fully invested near market bottoms -- with low cash reserves -- after snapping up bargains dumped on the market by panicky individual investors. Conversely, near market tops, they should sell heavily, accumulating large cash reserves by taking advantage of the speculative whims of an excited public. Again, theory and practice diverge widely. Rather than supporting stocks when prices plummet, they get trampled at the exit. When prices soar, they buy aggressively. The pros seem to judge market direction poorly.

Are investment advisory services that sell advice to the public by subscription any better? Many of these services, as they readily advertise, aim to get you in near the market bottom and out near the top. Would you have profited by subscribing to such services? Again no. For years, pundits, myself included, have publicly charted an almost perfect correlation between the investment advice given and the future direction of the market.

Unfortunately, it is almost perfectly wrong. When advisors go one way, markets go the other. As the averages approach their highs, larger and larger numbers of advisors become bullish, and as they move toward their lows, an increasing number stampede for the exits. The fact is so well known on Wall Street that many of us now use the advisory services as a contrarian indicator. If over 70% of services wax bullish -- look out -- markets are approaching a high. Conversely, if 70% are terrified of the bear, it's time to buy. This was true at the market high in 1972, when 71% of the services were bullish, at the bottom of the worst market break since the war in late 1974, when 70% thought the market was heading south, and just before the Great Bull Market began in the spring of 1982. Instead of the market following the advisors, the advisors seem to unerringly follow the market.

What then do we make of these statistics? Some skeptics and financial writers (the two being often difficult to distinguish) believe the figures indicate professional money management is inept. The most important service investment management could perform for clients, they might say, would be to walk their dogs or perhaps advise them on their business attire. Other skeptics have cheerfully suggested that blindfolded chimpanzees heavily fortified with margaritas could outperform the experts by throwing darts at the stock pages.

But humor aside, the knowledge that professionals do not outperform the market has been widely known for a generation. This is before considering the effects of the investment revolution. Adjusting for the new financial order, the results only get worse, putting the investor even further behind the eight ball. No, the odds of winning big do not appear to lie with this group, regardless of how much media attention they get or advertising they chum out.

I opened this chapter by assuring you that investors have an excellent chance of beating the market. Now we see even the experts cannot keep up with it. Something obviously is out of whack here, as financial academics began to realize several decades back.

A new world has since been discovered and mapped. The ivory tower explorers went sailing over seas of data, rather than uncharted oceans, but the attitude was similar. They were conquering new intellectual realms for the greater glory of academe, making sweeping pronouncements in the process, and coming up with unbeatable odds -- or so they said. Natives -- money managers, analysts, investors -- beware. You are about to be conquered.

The New Conquistadors

Some of the country's best academic minds looked at this problem soon after it was recognized in the early sixties. The financial professors found a radical new explanation of why professionals do not perform better. For a while, at least, this seemed to be the stock market's equivalent of the theory of relativity, the smashing of the atom, or maybe even the fountain of youth.

Using enormous computer power, by the standard of the day, they put forward an all-encompassing theory. Markets, they said, are efficient. That meant that stock prices are determined by the thorough and diligent work of the brightest analysts, money managers, and other investors. The combined knowledge of thousands of these experts kept prices exactly where they should be. No one can beat the market consistently. It might be hard to accept what the professors said, but they had proven this truth overwhelmingly. Besides, there were benefits. You no longer had to waste endless time and energy trying to outguess the market. No reason to get stressed out; it just couldn't be done. But you could still get higher returns. All you had to do was to take more risk.

So there it was. Forget the sweating, studying, and tossing and turning you used to do at night trying to buy the right stocks. It didn't help a hoot. Investing had been promoted to a science with the mathematical precision of quantum physics. Just sit back and enjoy the ride.

This major new creed has pervaded our academic institutions for three decades. Today, it affects almost every aspect of Wall Street thinking from the proper makeup of your retirement funds, to how to select an investment advisor. It also plays an important role in the decision-making of many large corporations, and influences the SEC and government policymakers. The hypothesis has been so widely accepted in academic circles and on Wall Street that several of its leading proponents -- Franco Modigliani, Harry Markowitz, William Sharpe, Merton Miller, Myron Scholes -- have received Nobel Prizes for their contributions.

The new theory came at a time when there was little to oppose it. The old market methods were dying. None of the ancient rituals seemed to work, no matter how rigorously they were applied, or how extensive the training, experience, or intelligence of the practitioner.

Into this vacuum came the seductive idea of efficient markets, offering a plausible explanation of the professionals' failure, absolving investors of blame (for it preached it was not in their power to change things). Order replaced chaos. Converts flocked to the new gospel by the millions.

The spread of the new faith was not unlike the conquest of the vast Inca empire by Francisco Pizarro and his 180 conquistadors. Like the conquistadors, the scholars used both the faith and the sword to annihilate the pagans' beliefs in the marketplace. If the true faith was not accepted, why then there was the sword -- the unleashing of volleys of awesome statistics disproving everything the professionals believed. The frightening new weapon of statistical analysis awed the Wall Street heathen more than Pizarro's cavalry did the Incas, who had never before seen a horse. What amazes, in looking back, is that the leaders of the new faith subdued millions of investors with a smaller troop than the original conquistadors.

But the golden age of efficient markets was not destined to last. Smash, boom, bang, along came the 1987 crash and the pillars of the new paradise crumbled. In retrospect, the elegant hypothesis had a minor hitch. The professors assumed investors were as emotionless and as efficient as the computers they used to build their theory. The October 19, 1987, crash, 80% worse than 1929 in percentage terms and over a thousand times larger in actual dollars, showed it just wasn't so. Like a bizarre Rube Goldberg machine, the theoretically flawless trading systems the professors had introduced turned into a monster unleashing massive panic. The market ended 508 points lower that day. In five trading days ending on October 19, the market had lost one-third of its value -- about one trillion dollars.

When the New York Stock Exchange (NYSE) opened on the morning of October 20, the professors' supposedly foolproof trading mechanisms sent stocks into another devastating free fall. To end the rout, the NYSE was forced to ban the new academic tools that were supposed to create Eden on Wall Street. The entire financial system came within a hair's breadth of disintegration, according to the Brady task force and SEC reports commissioned to study the crash. The worst crash in modern history was caused by a theory devoid of any understanding of investor psychology.

The academic dogma that investor psychology played absolutely no role in markets justified the new trading mechanisms responsible for the debacle. This was not something to be faulted only with 20/20 hind-sight. Numerous market observers warned of the impending calamity. Eighteen months before the crash, the new academic trading strategies caused Representative John Dingle of Michigan, then-chairman of the House of Representatives Committee on Energy and Commerce, which oversees the SEC, to fear a panic. John Phelan, then-president of the New York Stock Exchange, had also warned that the interaction of the new academic programs could cause a market meltdown. "When I first started talking about this in late 1986," Phelan was quoted as saying after the crash, "people would do almost anything but physically attack you."

I was also worried about the dangers inherent in the academic trading systems. Writing about them in a Forbes column six months before the crash, I noted that the enormous volume in computerized trading programs, portfolio insurance, and numbers of other such strategies all based on financial index futures at low margins, amounted to a potential doomsday machine. The column concluded that if uncontrolled, these aberrations set up the possibility of a sharp correction, if not a crash, not far down the road.

Even though the most destructive of the low-margined trading systems created by efficient market believers were all but completely dismantled in the next several years, the academics who sponsored them, like the faithful of other disproved scientific theories, hung on tenaciously. Everything was blamed but the actual reasons as determined by the commissions set up to study the causes of the crash. To do otherwise would be no less than admitting that the theory was as useful as the eighteenth-century medical belief that bleeding the patient balanced the body's humors.

No, the revolutionary new ideas that sprouted from ivory towers across the country won't give you the odds I've discussed earlier, or for that matter even keep you afloat. But though the theory began to disintegrate with the crash, it left an enormous void among investors. Many of the basic teachings of the efficient market hypothesis (EMH) have now have been conclusively refuted by advanced forms of the same statistical analysis that devastated the investment heathen. Yet contemporary investment practice is built around the belief in efficient markets. Large numbers of investors, though they believe the theory bankrupt, don't know where else to turn.

Where then do the odds of market success lie? Or are there any real odds at all?

A Titanic Clash

We find ourselves in the midst of a sea change of thinking in the investment world, one whose ramifications extend far beyond the stock market, and even beyond economics. Scientists call such changes paradigm shifts. A paradigm encompasses the working beliefs underlying all theory and research in a particular science. Paradigms shift rarely, and the shifts are always resisted by the creators of the old system of beliefs. Bitter polemics and enormous rancor are usually spilled before the change prevails. Still, it is within this paradigm shift that we will find the key to the winning odds in the marketplace.

It is never easy to give up your most basic beliefs. As we all know from our school days, the Ptolemaic system, which held that the earth was the center of the universe, resisted change for centuries. With improved telescopes and more accurate observation, sighting after sighting over the years found planets where the theory stated they shouldn't be. The model became increasingly complex in order to incorporate each new piece of information. The planets and stars moved around the earth in a combination of circles, epicycles, and deferents (points moving on large circles around which the epicycles simultaneously revolved). The result of this hodgepodge was a mind-boggling whirl.

Still, it took generations for scientists to accept the radically different heliocentric paradigm of planetary motion. As Paul Samuelson once phrased it, "Scientific progress advances funeral by funeral."

Markets whirl along in a similar state today. Each new statistical finding throws the efficient market hypothesis into further disarray and requires additional explanations for the sighting of "anomalies" ruled impossible by the theory.

That said, this book is not a scientific treatise. But it is important for the reader to have some understanding of the intellectual clashes in investments and economics today, because the winning odds we will examine, and the advice that stems from them, comes from a radically different paradigm than efficient markets, or for that matter conventional Wall Street thinking. In many ways, it is as different as Copernicus from Ptolemy.

What then do these other methods lack? On paper, at least, they have a high probability of success. The key principle of this book is that people are not the rational, omniscient decision-makers that the efficient market believers claim, and most investment practitioners believe. Rather, we are constantly pushed or pulled by psychological influences.

To be fair, it is not only these groups who do not understand investor psychology and make major mistakes as a result.

All of us are affected by these powerful but unrecognized influences.

All of us have heard of how investor psychology has turned positive or negative on an industry or company, or on the market as a whole.

All experienced investors have no doubt pondered the extreme psychological variations that markets display over time.

The problem we face is that, although we know much of the cause of success or failure lies largely within the realm of psychology, we really do not understand how and where psychology meshes with investment decisions. As a result, we make costly and sometimes disastrous errors time and again. Learning the principles underlying investment behavior will allow us to develop successful market strategies. The methods have worked well over long periods and should continue to succeed, because they are based on predictable patterns of investor behavior. These are the odds available in the green wing.

The psychology of market success that I'll describe is anything but "will-o'-the-wisp." Its bedrock is consistent and predictable investor errors in the marketplace. This bedrock will support a new theory of markets and investing. As you will see, the assumptions underlying the methods are well documented, both empirically and psychologically. At every stage of the development of our new strategies, I'll demonstrate the strong statistical proof the probabilities are built upon.

What we will see is that not only do investors go wrong, they go wrong in a systematic and predictable manner. So predictable, in fact, that consistent investment strategies can be built on their mistakes. More than that we can calculate odds on how well our strategies will work over varying periods, in a similar manner to the gambling casino. These strategies, as I'll demonstrate, have worked for generations, and I suspect since the dawn of markets. That is, if human behavior hasn't changed.

How and why these probabilities work is what this book is about. In examining them we will go far beyond markets, to some seemingly universal laws of human behavior. People encountering the same situation will act the same, within the marketplace or outside of it. Our new investment approach stands on market history, on the latest psychological research, and on thorough statistical analysis.

I hope to demonstrate that we can harness psychology and use it systematically to attain our investment goals. But before we attempt it, we must understand just how powerful these forces are, and why they lead most of us into the red rather than the green rooms. This is the reason why most investors, including the experts, underperform the market averages over time.

The Journey Ahead

The journey will be wide-ranging both in time and breadth. We will examine markets, psychology, finance, economics, statistics, even politics. Part of the text will naturally be somber, but it will have its lighter moments. I wrote in Psychology and the Stock Market over twenty years ago that "the integration of financial and psychological understanding is a new and basically uncharted field." To my amazement, the whole new field of behavioral finance has sprung up since. Psychologists, economists, and investment academics are now beginning to study markets and are finding anything but rational behavior. Their major new research can be of profound benefit to you. Once you understand the principles, there is nothing magical or mysterious about the work.

To return to that great probability player of warfare again, Napoleon once asked, "Is it because they are lucky that [great men] become great? No, but being great they have been able to master luck."

While no book can teach you how to make a "great" fortune, I believe this one can give you a much better chance of mastering your luck in markets. Once you see the odds, all you need is an open mind and the courage of your convictions. I am convinced that today the individual investor has an exceptional opportunity to beat both the market and the experts. If you are a serious investor, I think our journey will prove rewarding.