The officials on the Australian island left the camp two weeks earlier and halted the interviews with the asylum seekers, leaving them in doubt about a deal of resettlement with the US that was made during the Obama administration.
The abrupt departure of American officials from an Australian Pacific island refugee camp has fanned fears among asylum-seekers that plans to resettle them in the US may not go ahead, an activist group said on Sunday.
Canberra sends asylum-seekers who try to enter Australia by boat to camps on Nauru and Papua New Guinea's Manus Island, but the conditions there have been criticised by refugee advocates and medical professionals.
The Australian government struck a deal with Washington under former president Barack Obama to resettle some of those refugees in the US.
But doubts over the arrangement have persisted after President Donald Trump this year reportedly lambasted his Australian counterpart Malcolm Turnbull during a phone call and attacked it as a "dumb deal," before agreeing to go ahead with the proposal.
US Department of Homeland Security officials had been assessing the asylum-seekers at Nauru as part of the arrangement when they abruptly left the island on Friday and Saturday, Refugee Action Coalition spokesman Ian Rintoul said, days after the US passed its annual 50,000-refugee intake cap.
"They've (the DHS officials) given the people on Nauru no indication that they are coming back," Rintoul said.
About 200 refugees on Nauru have undergone interviews and medical check-ups, while on Manus, some 70 had been through a similar process, Rintoul said.
Australian Foreign Minister Julie Bishop said on Sunday she was confident the deal was still in place, adding that the "matter is progressing as we expected."
"We have been given assurances by President Trump and Vice-President Pence and others, that the agreement will be adhered to," Bishop told the Australian Broadcasting Corporation.
"And the [refugee cap] quota will roll over again on October 1."
The situation is particularly acute on Manus, with the camp set to close by October after a PNG Supreme Court ruling declared that holding people there was unconstitutional.
Australian Immigration Minister Peter Dutton has said those on Manus would not be moved to Australia and instead relocated to third countries such as the US and Cambodia or resettled in PNG.
"News like this makes us feel dead. It defuses the spark of hope that we try to hold on to," Manus refugee detainee Imran Mohammad, from Myanmar, said in a statement Sunday via Australia's Human Rights Law Centre.
More than 800 men are being held on Manus, and 370 men, women and children are detained on Nauru, according to Australian immigration data ending May 31.