Facebook, along with its Instagram and WhatsApp platforms, suffered a worldwide outage for hours, a day after a whistleblower accused Facebook of repeatedly prioritising profit over clamping down on hate speech and misinformation.
Facebook and its Instagram and WhatsApp services were back online after a massive and lengthy outage that added to the social network's woes.
Facebook's family of apps essentially "disappeared" from the internet for several hours after a traffic routing problem that made the sites unreachable by users, said Cloudflare on Tuesday, a website security company.
Around 5:45 pm ET, some Facebook users began to regain partial access to the social media app. WhatsApp continued to have connection problems for at least some people.
The outage began around noon Eastern time (1600 GMT) on Monday.
Facebook cited faulty configuration changes on its routers as the root cause of the nearly six-hour outage.
"Our engineering teams have learned that configuration changes on the backbone routers that coordinate network traffic between our data centres caused issues that interrupted this communication," Facebook said on Monday.
On Sunday, a whistleblower accused Facebook of repeatedly prioritising profit over clamping down on hate speech and misinformation. The firm owns Instagram and WhatsApp.
Shares of Facebook, which has nearly 2 billion daily active users, opened lower after the weekend whistleblower report and slipped further to trade down 5.3 percent in afternoon trading on Monday.
They were on track for their worst day in nearly a year, amid a broader selloff in technology stocks on Monday.
We’re aware that some people are having trouble accessing our apps and products. We’re working to get things back to normal as quickly as possible, and we apologize for any inconvenience.— Facebook (@Facebook) October 4, 2021
Internal mistake or sabotage by insider?
Facebook was inaccessible because users were not being directed to the correct place by the Domain Name System. Facebook itself controls the relevant settings, suggesting the problem was an internal one.
Security experts said the disruption could be the result of an internal mistake, though sabotage by an insider would be theoretically possible.
DNS allows web addresses to take users to their destinations. A similar outage at cloud company Akamai Technologies Inc took down multiple websites in July.
An outside hack was viewed as less likely.
A massive denial-of-service attack that could overwhelm one of the world's most popular sites would require either coordination among powerful criminal groups or a very innovative technique, security experts said.
See the latest stats from Downdetector on the global outage of Facebook, Instagram, WhatsApp and Facebook Messenger. https://t.co/OvrMQBOzr6— Downdetector (@downdetector) October 4, 2021
Facebook trying to get site working
Facebook acknowledged users were having trouble accessing its apps but did not provide any specifics about the nature of the problem or how many were affected by the outage.
"We're working to get things back to normal as quickly as possible, and we apologise for any inconvenience," Facebook tweeted about 30 minutes after the first reports of the outage.
One Facebook employee told Reuters news agency that all internal tools were down.
Facebook's response was made much more difficult because employees lost access to some of their own tools in the shutdown, people tracking the matter said.
Multiple employees said they had not been told what had gone wrong.
The social media giant, which is the second largest digital advertising platform in the world, was losing about $545,000 in US ad revenue per hour during the outage, according to estimates from ad measurement firm Standard Media Index.
The estimates were based on total Facebook and Instagram ad spending from major advertising agencies from January to August this year.
User reports indicate TikTok is having problems since 1:29 PM EDT. https://t.co/XTqQicEGuf RT if you're also having problems #tiktokdown— Downdetector (@downdetector) October 4, 2021
Similar widespread outages
Downdetector - which only tracks outages by collating status reports from a series of sources, including user-submitted errors on its platform - showed there were more than 50,000 incidents of people reporting issues with Facebook and Instagram.
The outage might be affecting a larger number of users.
WhatsApp, the social-media giant's instant messaging platform, was also down for over 35,000 users, while Messenger was down for nearly 9,800 users.
Facebook has experienced similar widespread outages with its suite of apps this year in March and July.
Several users using their Facebook credentials to log in to third-party apps such as Pokemon Go and Match Masters were also facing issues.
"If your game isn't running as usual please note that there's been an issue with Facebook login servers and the moment this gets fixed all will be back to normal," puzzle game app Match Masters said on its Twitter account.
Susceptible to social engineering
So many people are reliant on Facebook, WhatsApp or Instagram as a primary mode of communication that losing access for so long can make them vulnerable to criminals taking advantage of the outage, said Rachel Tobac, a hacker and CEO of SocialProof Security.
"They don't know how to contact the people in their lives without it," she said.
"They’re more susceptible to social engineering because they’re so desperate to communicate."
Tobac said during previous outages, some people have received emails promising to restore their social media account by clicking on a malicious link that can expose their personal data.
Jake Williams, chief technical officer of the cybersecurity firm BreachQuest, said that while foul play cannot be completely ruled out, chances were good that the outage is "an operational issue" caused by human error.
Facebook and Instagram down pic.twitter.com/T22UiaFqZX— Daniel Newmaη (@DanielNewman) October 4, 2021
'A world of hurt'
Computer scientists speculated that a bug introduced by a configuration change in Facebook’s routing management system could be to blame.
Columbia University computer scientist Steven Bellovin tweeted that he expected Facebook would first try an automated recovery in such a case.
If that failed, it could be in for "a world of hurt" — because it would need to order manual changes at outside data centres, he added.
"What it boils down to: running a LARGE, even by Internet standards, a distributed system is very hard, even for the very best," Bellovin tweeted.
hello literally everyone— Twitter (@Twitter) October 4, 2021
Twitter, meanwhile, chimed in from the company's main Twitter account, posting "hello literally everyone" as jokes and memes about the Facebook outage flooded the platform.
Later, as an unverified screenshot suggesting that the facebook.com address was for sale circulated, Twitter CEO Jack Dorsey tweeted, "how much?"