Global Facebook Services Outage
Facebook recently had a terrible outage that caused all its services to go offline globally. The incident caught attention as the users flooded Twitter with WhatsApp, Instagram, and Facebook’s inaccessibility complaints.
Many people initially suspected it to be an issue with their ISP, it turned out that the issues appeared at Facebook’s end. Consequently, Facebook, WhatsApp, Instagram, and Oculus remained unavailable for over six hours, causing massive trouble.
It is pertinent to mention that Facebook services hold a necessary stance for business and individual users for their day-to-day communications and business activities. Hence, these outages were frustrating for some users.
Finally, Facebook authorities confirmed the glitches via their official Twitter accounts.
We’re aware that some people are having trouble accessing our apps and products. We’re working to get things back to normal as quickly as possible, and we apologize for any inconvenience.
— Facebook (@Facebook) October 4, 2021
Instagram and friends are having a little bit of a hard time right now, and you may be having issues using them. Bear with us, we’re on it! #instagramdown
— Instagram Comms (@InstagramComms) October 4, 2021
We’re aware that some people are experiencing issues with WhatsApp at the moment. We’re working to get things back to normal and will send an update here as soon as possible.
Thanks for your patience!
— WhatsApp (@WhatsApp) October 4, 2021
After all the chaos, Facebook has finally revealed what triggered the issue.
As elaborated in a blog post from Santosh Janardhan, Facebook’s VP, Infrastructure, the outages happened due to a technical issue during “configuration changes on backbone routers”.
Our engineering teams have learned that configuration changes on the backbone routers that coordinate network traffic between our data centers caused issues that interrupted this communication. This disruption to network traffic had a cascading effect on the way our data centers communicate, bringing our services to a halt.
This problem affected not only the services but also the tech giant’s internal systems. Hence, it became difficult for the teams to resolve the matter.
According to reports, even Facebook employees couldn’t enter their workplaces as their badges to access the doors stopped working.
Was just on phone with someone who works for FB who described employees unable to enter buildings this morning to begin to evaluate extent of outage because their badges weren’t working to access doors.
— Sheera Frenkel (@sheeraf) October 4, 2021
What Facebook Disappeared From The Internet
Although Facebook’s explanatory post doesn’t reveal many technical details about why exactly it suddenly disappeared from the internet. What people observed during the outages was that visiting “facebook.com” would return no results. Rather the domain appeared to be ‘for sale’ (due to a third-party mess-up unrelated to Facebook’s technical issue).
Explaining the matter, Johannes B. Ullrich from the SANS Technology Institute highlighted the problem as a DNS error. As elaborated,
The BGP routes pointing traffic to Facebook’s IP address space have been withdrawn. The Internet no longer knows where to find Facebook’s IPs. One symptom is that DNS requests are failing. But this is just the result of Facebook hosting its DNS servers inside its own network. Even with working DNS (for example if you still have cached results), the IPs are currently not reachable.
Service Restoration Underway
Hours after the outages, Facebook officials eventually started restoration efforts (of course, after getting into the data centers, finally). As Janardhan explained in the post, the team is actively working for 100% service restoration. He also ruled out any cyber attacks behind this situation.
We want to make clear at this time we believe the root cause of this outage was a faulty configuration change. We also have no evidence that user data was compromised as a result of this downtime.
Indeed, many users worldwide can now access their Facebook, Instagram, and WhatsApp accounts again. Anyone still facing issues should rest assured to get the access back shortly.
Let us know your thoughts in the comments.