"Don't call my bluff," the president told Cantor, according to the GOP leader.

Democratic and Republican sources have offered several versions of the final moments of Wednesday's meeting -- by most accounts it ended on a low note, raising questions about whether negotiators can make any progress under increasing pressure to strike a deficit-reduction deal and raise the debt ceiling. A startling warning came Wednesday when credit rating agency Moody's said the U.S. government's impeccable credit rating was under review for a potential downgrade. The agency said the ongoing stalemate increased the risk of a first-ever default on the government's debt.

Leaders from each side cited the announcement from Moody's, calling on the other to give ground for the sake of avoiding fiscal calamity. But the area of agreement between the two sides seems only to have gotten smaller in the past few days.

One Republican aide told Fox News that negotiators are "walking back" the value of proposed spending cuts -- from $2 trillion to $1.5 trillion. Cantor apparently tried to suggest negotiators strike a short-term deal considering the distance between the two sides, but Obama shut that idea down.

According to sources, Obama said "enough is enough," arguing forcefully for revenue increases to achieve the deficit reduction sought by the GOP.

"I've reached my limit. This may bring my presidency down, but I will not yield on this," Obama said, before walking out.

"He became very agitated," Cantor said. He called the president's exit from the talks "abrupt," and added that the two sides are "very far apart right now."