Investment in buy-to-let plummeted to just £5bn in 2017 having been worth £35bn ($48.8bn, €39bn) in 2015.
Changes to the tax treatment and mortgage requirements are thought to be behind the slump, which IMLA branded “excessive” and warned against “further punitive action”.
As a result of the tax changes, 21% of landlords said they would be reducing their portfolio in the coming year.
According to IMLA, buy-to-let landlords have been good for UK housing with adjusted rental costs in real terms falling 4.4%.
IMLA executive director Kate Davies said: “The raft of regulatory and tax changes that have hit the buy-to-let market in the last year have far-reaching effects that are still yet to be fully realised.
“We know that the majority of people regard owner-occupation as the tenure of choice, but for many this is not an immediate option. We also know that those who would, in the past, have rented from their local authority or housing association now need to rent privately.
“We urge the government to reassess the impact of the recent far-reaching regulatory changes to buy-to-let investment and allow a period of policy consolidation.
“Our nation’s private rental sector investors provide a vital service that’s vital to millions of UK tenants. We need to support and protect a sector that does so much for so many.”