Algerian special forces have freed 650 hostages from Islamist militants who seized a gas complex deep in the desert, Algerian state media said Friday, but the fate of some foreign workers remains unclear.

Of those taken hostage Wednesday, 573 were Algerians, according to the state-run Algerian Press Service news agency. It said “over half” of the 132 foreign workers held in the hostage crisis have been freed, according to a provisional count.

CNN has not independently confirmed the APS report.

Details of the total number of hostages seized at the In Amenas installation and their nationalities are still not clear.

Some foreign workers have sought refuge in various parts of the Saharan site, APS cited a security source as saying.

The special forces are still trying to reach a “peaceful ending,” the source told APS, before “neutralizing” the terrorist group. The group is made up of about 30 militants of different nationalities, security sources told the news agency.

The remote gas field, about 60 kilometers (37 miles) west of the Libyan border and 1,300 kilometers (about 800 miles) from the Algerian capital, Algiers, is run by Algerian state oil company Sonatrach in partnership with Britain’s BP and Norway’s Statoil.

State-run Algerian Radio earlier cited an official source as saying that a major military raid launched Thursday was over but that there was “ongoing activity at various locations” near the plant.

British Prime Minister David Cameron said Friday morning that the Algerians were still pursuing terrorists and possibly hostages at the large and complex site.

The number of Britons unaccounted for is “significantly” fewer than 30, he told the House of Commons, but he declined to give an exact number because of the fluidity of the situation.

Britain was not informed in advance of the Algerian military operation Thursday, Cameron said. But he stressed that it was the militants — whose leader was linked to an al Qaeda offshoot — who were to blame for the situation.

Cameron said nationals from at least seven other countries were involved, as well as many Algerians.

U.S. Defense Secretary Leon Panetta, speaking in London, said the United States was working round the clock to ensure the safe return of its citizens.

“Regardless of the motivation of the hostage takers, there is no justification for the kidnapping and murder of innocent people,” he said. “Terrorists should be on notice that they will find no refuge, not in Algeria, not in North Africa, not anywhere. Those who attack our country or our people will have no refuge.”

U.S. authorities have not specified how many Americans are caught up in the attack. Officials from Norway, France, Malaysia and Japan have said their nationals are also among those involved.

And after a day of chaotic reports Thursday, many relatives of workers at the besieged plant are braced for bad news.

The U.S. is evacuating between 10 and 20 people caught up in the hostage-taking, a U.S. defense official told CNN on Friday.

They will be taken to U.S. facilities in Europe, the official said, and the condition of those who are injured will be assessed. “We just don’t know what kind of injuries they have,” he said.

BP said Friday that a “small number of BP employees” are still unaccounted for, while Statoil said the fate of eight of its employees at In Amenas was still uncertain. Nine other Statoil workers who were at the plant are safe, it said.

Both firms are pulling personnel from Algeria. BP said it had flown 11 of its own employees and several hundred staff from other companies out of the North African country Thursday and expected another flight Friday.

Britain has deployed a diplomatic “rapid reaction” plane to a location about 280 miles from the besieged plant, a Foreign Office official said.

On board are trauma experts and consular affairs officers who can issue emergency passports. “It’s the kind of thing we have done before in similar situations,” the official said. “This is us getting as close and as ready as we can.”

A Briton was one of two people confirmed killed Wednesday.

Algeria faces tough questions from the governments of kidnapped nationals over its handling of the crisis amid fears that hostage safety is not being put first.

Japanese Vice Minister Shunichi Suzuki summoned the Algerian ambassador Friday to express Tokyo’s concern, the Japanese Foreign Ministry said. And Japanese Prime Minister Shinzo Abe is cutting short a foreign trip to deal with the crisis, his office said.

“There is so much conflicting information on safety of the hostages,” Chief Cabinet Secretary Yoshihide Suga told reporters in Tokyo. “Safety of 14 Japanese citizens still remains unknown.”

He said Japanese officials had urged the Algerian government to avoid exposing the hostages to danger. “We are terribly disappointed about the Algerians’ military operation,” he said.

Before the Algerians launched the military operation, U.S. officials urged them to be cautious and make the hostages’ safety their first priority, an Obama administration official said.

Algerian government officials did not warn their U.S. counterparts about the raid, the official said.

A senior U.S. official said U.S. officials did not trust the information they were getting from the Algerians, “because we hear one thing and then we hear something else.”

Algerian forces went in Thursday after noticing the hostages being moved toward “a neighboring country,” where kidnappers could use them “as a means of blackmail with criminal intent,” Algerian Communications Minister Mohamed Said told state television.

Algerian troops fired on at least two SUVs trying to leave the facility, Algerian radio said. And a reporter saw clashes near the site, the Algerian Press Service and radio reports said.

“There were a number of dead and injured, we don’t have a final figure,” the communications minister said.

Former hostage Stephen McFaul, who escaped Thursday, gave a dramatic account of his ordeal to his brother, Brian McFaul.

Stephen had plastic explosives strapped around his neck, duct tape over his mouth and rope around his hands, Brian McFaul told CNN from Belfast.

His brother made a break for freedom after the vehicle he was in — one of several targeted by Algerian fighters — crashed, with the explosives still around his neck, he said.

“The joy was unreal,” Brian McFaul said upon hearing that his brother was safe. “I haven’t seen my mother move as fast in all my life, and my mother smile as much, hugging each other. … You couldn’t describe the feeling.”

A British Foreign Office spokeswoman said that the country was in close communication with the Algerian Foreign Ministry and that all its efforts were focused on getting the hostages out safely.

On Thursday, Algeria’s state media reported that all Algerian nationals who had been held hostage were free: some had fled, while others were released.

In total, nearly 600 workers and four foreign nationals — two Scots, a Kenyan and a French citizen — were free by late Thursday, APS reported. It said there were numerous casualties in the Algerian military operation.

The chief executive of a catering firm on the site, which had 150 hostages freed, told CNN that before the military raid some hostages “disguised” themselves to escape.

Regis Arnoux, of CIS, described the hostages as “traumatized.” The freed hostages told him that all Westerners were rounded up in one group and tied up together at the beginning of the siege. When the militants realized the Algerian forces had surrounded the site, the group was divided up.

He told CNN that some Algerian hostages were free to walk around the site but not to leave. Some managed to escape by themselves.

The militants’ assault early Wednesday was carried out apparently in response to France’s offensive in neighboring Mali.

The kidnappers were equipped with AK-47 rifles and put explosives-laden vests on some of the hostages, a U.S. State Department official said.

On Wednesday, the attackers put the number of hostages at “more than 40,” including seven Americans, two from France, two Britons and other Europeans. Another Islamist group told the Mauritanian News Agency there were 41 “Westerners.”

Americans are among the hostages at the facility, White House spokesman Jay Carney said, without specifying how many. There could be as few as three American hostages, two U.S. officials said Wednesday.

One of the kidnapped Americans is a Texas man, a family member told CNN.

By Thursday night, some Americans had been freed and had spoken with relatives back home, while others remained unaccounted for, U.S. officials said.

“This incident will be resolved — we hope — with a minimum loss of life,” U.S. Secretary of State Hillary Clinton said. “But when you deal with these relentless terrorists, life is not in any way precious to them.”

Three workers for a Japanese engineering company that was working on the site have been contacted and are safe, said Takeshi Endo, a senior manager for JGC Corp. But the company had not been able to contact 14 others, he said.

CNN affiliate BFM-TV reported that a French nurse who was working at the site at the time of the attack had been freed.

The man behind the group claiming responsibility for the attack and kidnappings is a veteran jihadist known for seizing hostages.

Moktar Belmoktar, an Algerian who lost an eye fighting in Afghanistan in his teens, has long been a target of French counterterrorism forces. Libyan sources said he spent several months in Libya in 2011, exploring cooperation with local jihadist groups and securing weapons.

The militants said they carried out the operation because Algeria allowed French forces to use its airspace in attacking Islamist militants in Mali. Media in the region reported that the attackers issued a statement demanding an end to “brutal aggression on our people in Mali” and cited “blatant intervention of the French crusader forces in Mali.”

French Prime Minister Jean-Marc Ayrault said the Algerian hostage situation “confirms the gravity of the terrorist threat and the necessity to fight it with a determined and united front.”

The death of several hostages is to be deplored, he said, adding that neither the “precise number of victims nor their nationalities” are known.

—CNN’s Barbara Starr, Dan Rivers, Mitra Mobasherat, Saskya Vandoorne, Laura Perez Maestro, Junko Ogura, Dheepthi Namasivayam, Saad Abedine, Elise Labott, David Mattingly, Athena Jones, Jethro Mullen, Tim Lister, Joe Sterling and Greg Botelho contributed to this report, as did journalists Peter Taggart from Belfast and Said Ben Ali from Algiers.