Facebook has attempted to explain the issues that caused an unprecedented shut down of its major apps and services on Monday.
The main Facebook social network, Messenger offshoot, Instagram and WhatsApp properties were out of commission for more than seven hours for users around the world on October 4.
After issuing an apology last night, Facebook has updated users on what happened, in an effort to limit speculation about a potential malicious attack on the under-fire company, or the potential loss of user data.
In a post on the Facebook engineering blog, the company blamed the outage on the system that manages the global backbone network capacity. The result was the spine that connects all of Facebook’s data centres around the world completely shitting down.
“The backbone is the network Facebook has built to connect all our computing facilities together, which consists of tens of thousands of miles of fibre-optic cables crossing the globe and linking all our data centres,” wrote Santosh Janardhan, who is the company’s vice president in charge of infrastructure.
The firm goes into much more detail, saying the DNS problems first suspected to be causing the issue were a result of the backbone collapse. The DNS addresses disappeared from the web yesterday, effectively making Facebook unreachable by anyone.
“To ensure reliable operation, our DNS servers disable those BGP advertisements if they themselves can not speak to our data centers, since this is an indication of an unhealthy network connection,” Janardhan continued. “In the recent outage the entire backbone was removed from operation, making these locations declare themselves unhealthy and withdraw those BGP advertisements. The end result was that our DNS servers became unreachable even though they were still operational. This made it impossible for the rest of the internet to find our servers.”
The company says that once the issue was identified, it was able to bring the backbone back online, thus restoring the functionality.
It’s been an incredibly rough week for the Silicon Valley giant thus far, with US Congress investigating the harmful effects of its platforms on younger viewers.