One in five people fear having to move out of their local area because of the high cost of housing - and younger generations appear to be the most affected, according to a new survey.
The Chartered Institute of Housing (CIH), which commissioned the poll, said the findings suggested the country was in the grip of a housing crisis which is "causing misery for millions".
And CIH chief executive Grainia Long called on politicians to take urgent action to tackle the problem by increasing the supply of homes.
Ms Long said: "These figures are further confirmation that our housing crisis is causing misery for millions of people across Great Britain and fuelling fears for the future - particularly among younger people."
She went on: "All parties must do much more to convince the people of Great Britain that they can address the issue.
"Now is the time for action - we're challenging all parties to put tackling the housing crisis front a centre in their manifestos.
"To have any hope of tackling our housing crisis we must drastically increase the number of new homes we build across all tenures."
The online survey by Ipsos MORI involving 2000 people aged 16 to 75 found that 40% of people would be in favour of more homes being built in their area.
It found that 20% of people believe they may have to leave their local area because the cost of housing is too high. Young people are the most concerned, with 36% of 16 to 24-year-olds and 26% of 25 to 34-year-olds believing they will have to move out of the area.
More than half of 16 to 24-year-olds were worried about rising house prices, with 52% saying that they would be a bad thing for them personally, compared with 32% across Great Britain as a whole.
Many are concerned about being able to keep up their rent or mortgage payments, with 24% of people and 32% of 16 to 24-year-olds being worried about this. Some 29% of those surveyed are concerned they may not be able to meet their mortgage payments in a year's time.
The survey also revealed that 60% of people believe it is harder for them to buy than it was for their parent's generation. A total of 32% said that their housing costs were causing them stress.
When asked, 52% of people said that no political party had the best policies on housing, and 41% believed that the parties did not pay a lot of attention to housing.
The survey was released in a week in which the National Association of Estate Agents revealed that 19% of all properties sold in May went for more than the asking price as demand increases and people face fierce competition for houses.
Latest official figures also showed that house prices leapt by 9.9% over the last year to reach a new high of £260,000 typically in April.
CIH, which is the body representing housing professionals, released the figures ahead of Housing 2014, its annual conference and exhibition, which takes place at Manchester Central from June 24-26.
Housing Minister Kris Hopkins said: "This government is fixing the broken housing market we inherited in 2010. We've scrapped the failed top-down planning system, built over 445,000 homes, and kept interest rates low for homeowners. We've also helped homebuyers get on the housing ladder, because if people can buy homes, builders will build them.
"House building and buying are now at their highest level since 2007 and climbing, while rents are falling in real terms. Last year councils gave permission for 216,000 new homes under the locally-led planning system, and more than 1,000 communities have swiftly taken up neighbourhood planning.
"It's clear evidence the Government's long-term economic plan is working, but there is still more to do, and we will continue to prioritise resources to build more homes and improve the housing market."