Both camps in Nauru and Manus island have been criticised by the United Nations and several other human rights groups for blatant human rights violations.
Australia and Papua New Guinea announced on Wednesday they had agreed to close the detention center located in Manus Island that is hosting 800 asylum seekers.
The island is located in Papua New Guinea (PNG) and has been the home of refugees trying to relocate in Australia for some years now.
However, under Australian law, anyone caught trying to reach the country by boat is sent either to Nauru or to Manus Island off PNG and can never be resettled in Australia.
Both camps in Nauru and Manus island have been criticised by the United Nations and several other human rights groups.
The complaints came after more than 2,000 reports of sexual abuse, assault and self-harm at the Nauru detention center were leaked by a newspaper.
"Both Papua New Guinea and Australia are in agreement that the centre is to be closed," PNG Prime Minister Peter O'Neill said in a statement.
"A series of options are being advanced and implemented. It is important that this process is not rushed out but carried out in a careful manner."
Australian Immigration Minister Peter Dutton met with O'Neill on Wednesday where he took the opportunity to reiterate Australia's position on not accepting the relocation of any of the refugees detained in PNG.
"It has been the longstanding position of this government to work with PNG to close Manus and support those people as they transition into PNG or return to their country of origin," Dutton said in a statement.
"Our position, confirmed again today with PNG, is that no one from Manus Island Regional Processing Centre will ever be settled in Australia."
Australia Director for Human Rights Watch tweeted a quote from her statement on the issue saying the refugees should be moved either to Australia or another safe country.