As we learned during the economic downturn of the past several years, being able to retire when you want and with enough money to live comfortably is far from a sure thing. And a new survey shows Americans' expectations for retirement remain low.

According to the Employee Benefit Research Institute, 28% of workers have no confidence they'll be able to afford a comfortable retirement. That's the highest level in the survey's 23-year history.

It's hardly surprising when you dig into the numbers. Fifty-seven percent of those surveyed report having less than $25,000 in savings and investments (not counting the value of their homes and any defined benefit pension plans.) Just 66% of workers say they or their spouse are saving for retirement, down from 75% in 2009. A mere 2% of workers say retirement saving and planning are their most pressing financial issues.

And then there's debt. More than half of workers, as well as 39% of people already retired, say they owe too much. And that's hampering their ability to deal with short-term financial emergencies, much less long-term goals like retirement savings.

“Only about half of workers and a comparable number of retirees say they could definitely come up with $2,000 if an unexpected need arose within the next month,” said Matt Greenwald of Greenwald & Associates, which conducted the survey.

Perhaps most troubling is that many workers are flying blind when it comes to retirement. Only 23% say they've sought advice from a financial advisor, and 45% admit guessing how much they'll need to save for retirement, instead of seeking advice from a pro or using an online calculator.

Getting the numbers right is critical; with life expectancy increasing, so is the likelihood that people will outlive their money. Although day-to-day expenses can get in the way of long-term savings goals (41% in the survey cited cost of living as a reason they weren't contributing, or contributing more, to their retirement plan) it's important to fund that nest egg.

HLN money expert Clark Howard says you should put at least 10% of your income toward retirement. If that's too ambitious, then set aside what you can, even if it's only 1%. Bump it up another 1% every six months. If you're in an employer-sponsored plan, contribute at least enough to get the match if one is offered -- it's free money.