For more than two hours on Thursday, a large chunk of mobile Internet traffic in Europe was rerouted through the network of China Telecom, thanks to a significant BGP route leak that affected several mobile carriers.
BGP route leaks are a constant problem on the Internet and have been since the network was first cobbled together. The Border Gateway Protocol (BGP) is one of the fundamental protocols that makes the network run and is how routers in one network tell routers elsewhere what the best route to a certain destination is. Each individual network publishes route information about the optimal route to reach the network, and other routers across the Internet pick up that route information. If a route breaks for one reason or another, the network can publish a different route to ensure continued traffic flow.
But because routers don’t verify that the routes announced by a given network are correct, a network owner can announce a route that belongs to another network and wind up with traffic that was intended for another destination. Some of these incidents are malicious, but many others are accidents, and Thursday’s incident looks like the latter, though it’s always difficult to tell with BGP leaks. The incident began early Thursday, when Safe Host, a Swiss hosting provider, leaked more than 70,000 BGP routes to China Telecom, which then announced those routes as its own, which meant the rest of the Internet then assumed they belonged to China Telecom.
The result was traffic from a number of mobile carriers in France, the Netherlands, and Switzerland running through China Telecom’s network on the way to wherever it was meant to go. That’s not the way that the Internet is supposed to work, but unfortunately is one of the side effects of its architecture. Traffic usually goes where it’s supposed to, but there may be some unscheduled stops along the way.
“Today’s incident shows that the Internet has not yet eradicated the problem of BGP route leaks. It also reveals that China Telecom, a major International carrier, has still implemented neither the basic routing safeguards necessary both to prevent the propagation of routing leaks nor the processes and procedures necessary to detect and remediate them in a timely manner when they inevitably occur,” said Doug Madory, director of Internet analysis at Oracle, who monitored the leak.
BGP leaks and and BGP hijacking incidents are quite common, but they’re typically shorter and smaller in scope than Thursday’s leak. More often, a leak will affect one specific organization.
“Two hours is a long time for a routing leak of this magnitude to stay in circulation, degrading global communications,” Madory said.
Experts say BGP leaks, while distressingly common, are also preventable in some cases, and, thankfully, quite visible to the network.
“We continue to see periodic cases where the inherent lack of security built into BGP leads to anomalous routing events. There isn’t currently much an enterprise can do to prevent someone advertising more attractive routing to their prefixes than the legitimate owners, but these events are highly visible and can be monitored,” said Patrick Sullivan, senior director of security strategy at Akamai.
“There are often questions about the root cause of these events, some are caused by engineers making erroneous changes to configs, while others appear more suspicious and are almost certainly committed deliberately.”