THE former prison in Victoria Road, Douglas, could be demolished.
The Department of Home Affairs has lodged a planning application for demolition but is waiting to hear from the planning authority about whether the scheme can go ahead.
It has been more than two years since the prison closed and all prisoners were transferred to a newly built prison in Jurby.
Home Affairs Minister Adrian Earnshaw said: "My department has received many requests from people interested in using the old prison for various projects which we have accommodated whenever possible.
"However a recent survey indicates parts of the gatehouse and cell blocks have deteriorated to a degree where the risk associated with allowing public access is something I am no longer comfortable with.
"Therefore the department is reluctantly having to refuse all new requests for access.
"The building occupies a prime two acre site in Douglas and I now want its fate to be decided as soon as possible.
"There are surprisingly significant costs in maintaining this Victorian building without repairs to its fabric. Those costs will only increase and that is why I want to see the future of the former prison resolved."