Are stocks overvalued? Depends on the tea leaves

After a healthy bull run since the Great Recession ended five years ago, some market watchers are warning that stock investors are in for a much rougher ride.

That's because some key measures of stock values are flashing signs that stock prices may have gotten ahead of themselves. A lot, though, depends on how you read the tea leaves, and which leaves you choose to look at.

Some of the strongest warnings that stocks are overvalued lately have been coming from Nobel Prize-winning economist Robert Shiller, who told CNBC earlier this month that he sees the "risk of substantial declines" ahead.

Shiller bases his bearish view on a long-term analysis of the relation between stock prices and earnings, the basic driver of stock values. In order to smooth out the effects of short-term market gyrations, Shiller takes the total earnings for the companies in the widely watched Standard & Poor's 500 index, averages them over 10 years, and adjusts for inflation. The result is a measure called the cyclically adjusted price-to-earnings, or CAPE, ratio. And the CAPE, he said, is pointing to a big drop in the overall stock market.

Shiller figures the current CAPE ratio—even after the recent market pullback—is around 25. That's much higher than the historic average of around 17, which means that stock prices would have to fall sharply for the ratio to revert back to that average level. If it did, stocks would fall by some 30 percent, with the Dow Jones industrial average dropping to about 11,000 and the S&P 500 diving to about 1,300.

Shiller's not saying that's going to happen soon, just that the CAPE ratio is flashing a warning signal.

"I have a general bias towards down because the market is overpriced, but these things unfold over years," he said.

As examples, he points to a CAPE reading that peaked at 44 in 2000—just before the Internet bubble burst. After rising again in the 2000s, the CAPE ration peaked at 27 in 2007 before the market staged another big pullback.

But while current CAPE readings are substantially higher than their long-term average, that average extends back more than 140 years, during a period with the U.S. economy, corporate taxes and stock investing was very different than today. That's why some market analysts argue the CAPE index should be taken with a large grain of salt.

Read More Market 'aftershocks' are coming: Robert Shiller

The use of a 10-year average profit reading, for example, understates stock values because economic downturns have gotten shorter and expansions are longer in recent years, according to Minnesota State University finance professor Stephen Wilcox.

"The problem with using a moving average that is longer than the business cycle is that it will overestimate 'true' average earnings during a contraction and underestimate 'true' average earnings during an expansion," Wilcox wrote in a 2011 AAII Journal article.

CAPE critics also argue that long-term changes in U.S. accounting methods, corporate taxes and reporting rules for earnings make it impossible to compare today's profit levels with those from 100 years ago. As a result, they argue, investors shouldn't put too much stock in the ratio's long-term average.

There are also other measures that present different ways to benchmark of stock values.

One is the government's measure of corporate profits contained in the quarterly data the Bureau of Economic Analysis collects to track the growth of the overall gross domestic product. By comparing that measure to the overall stock market's value (as measured by the Wilshire 5000 stock index), a somewhat different picture emerges.

In the run up to the 2000 Internet bubble, stock prices continued to surge even as overall after-tax profits were falling. In the early 2000s, stock prices and earnings came back into line, rising in the early half of the decade and then falling sharply as the Great Recession took hold.

Since then, profits have led stock prices higher. But corporate profits have largely stalled in the last three years, and have fallen since the start of the year even as stock prices continue to rise.

But stocks change hands based on more than just the latest quarterly profit report. A lot depends on what other alternative investments are available. And with government bonds stuck at historical lows, some market watchers use that alternative as a benchmark for the outlook for stock prices.

One way to value stocks based on Treasury bonds is to compare the returns on those investments. With the 10-year Treasury bonds currently yielding a little over 2 percent, stocks are offering an "earnings yield" of nearly 5 percent, which makes stocks undervalued based on current market prices.

"Of course, this 'Fed Model,' as I first named it back in July 1997, has been showing that stocks are undervalued since the tech bubble burst," said market analyst Ed Yardeni, president of Yardeni Research, in a blog post earlier this year.

Investors also value stocks based on the size of the dividend they pay, a reliable income source that for generations often represented the bulk of a stock's return.

But comparing current market prices based on long-term average dividend yields can also be problematic, thanks to a shift in the way many companies reward their shareholders. For a variety of reasons, cash payouts have been replaced in part by stock buybacks, with corporate managers using profits to buy back their company's stock to drive up share prices.

"In this scenario, the source of irrational exuberance is the ultra-cheap money available in the bond market for share buybacks and (mergers and acquisitions) thanks to the ultra-easy monetary policies of the Fed," wrote Yardeni.

If, as many market watchers expect, long-term interest rates began rising back to more historical levels, that could spell trouble for stock valuations based on "ultra-cheap money."

More From CNBC

• Top News and Analysis
  

• Latest News Video
  

• Personal Finance