Thirty-three out of 4,000 prisoners in England and Wales eligible for early release to help tackle coronavirus have so far been freed, ministers have said.
Justice Secretary Robert Buckland said progress on a scheme announced at the start of the month had been “slow”.
But he said other measures, such as requiring inmates to spend more time in their cells, had so far helped to prevent an “explosive outbreak”.
But Labour’s David Lammy warned of “rising tensions” in the prison estate.
The shadow justice secretary said forcing prisoners to spend “23 hours a day in their cells” could not continue indefinitely and a huge increase in testing was needed.
Coronavirus cases have been confirmed in more than half the prisons in England and Wales. A total of 321 prisoners have so far tested positive, and 15 inmates have died.
Speaking in Parliament, Mr Buckland told MPs that while each death was a matter of regret and prisons were “not out of the woods yet” the numbers were lower than many had predicted at the start of the outbreak.
As one of a number of measures aimed at containing the spread of the virus, up to 4,000 prisoners who were due to be freed within two months anyway were potentially earmarked for early release programme on 4 April.
The scheme was suspended just over a week ago after six offenders were released by mistake.
At the time, officials said processes had been tightened up and hundreds would be freed by the end of the week, including 14 pregnant prisoners and mothers with babies.
Pressed on the latest figures, Mr Buckland said “progress, I admit, has been careful and slow” but insisted that no-one was being freed before the end of their term without a full risk assessment.
“We have reached a point where, also taking into account the release of pregnant women, a total of 33 prisoners have been released.”
He said he had not “embarked on the scheme lightly” and it had to be judged in tandem with other measures designed to reduce total prison numbers and to allow for greater “compartmentalisation” within jails – such as freeing up additional space and greater use of non-custodial sentences.
Under the “restricted regime” currently in force, he said prisoners were spending more times in their cells “to support social distancing” and taking exercise and showers in small, supervised groups.
He said prison governors must continue to adhere to this guidance.
In response, Mr Lammy said “just 33 out of a figure of up to 4,000” was not encouraging and questioned what the government’s exit strategy was for trying to return prisons to normal.
“He will recognise that we cannot keep prisoners in their cells for 23 hours a day. It puts prison staff at risk, never mind potentially breaching very serious human rights.”
Campaigners have threatened to take legal action against the government unless vulnerable and elderly prisoners are immediately released to protect them from contracting coronavirus.
Lib Dem MP Christine Jardine said all prisoners on short-term sentences should be considered for release and no-one sentenced to less than a year should be imprisoned.
And ex-justice minister Bob Neil said ministers must ward against complacency about a future spike in cases
“Although the rates of infection and death are mercifully much lower than expected, none the less very great strains are placed, particularly on overcrowded local Victorian jails.
He urged ministers to use “all measures to maintain its responsibilities in domestic and European law to protect the welfare of prisoners and staff.”