T-Mobile resolves 13-hour outage, cites IP traffic issue

T-Mobile finally fixed network issues that had caused intermittent voice and text outages that affected customers across the US. The outages, which started June 15 just after 12 p.m. ET and continued for about 13 hours, were an “IP traffic related issue” that “created significant capacity issues in the network core throughout the day,” according to an update from T-Mobile CEO Mike Sievert. Later, T-Mobile President of Technology Neville Ray wrote:

The trigger event is known to be a leased fiber circuit failure from a third party provider in the Southeast. This is something that happens on every mobile network, so we’ve worked with our vendors to build redundancy and resiliency to make sure that these types of circuit failures don’t affect customers. This redundancy failed us and resulted in an overload situation that was then compounded by other factors. This overload resulted in an IP traffic storm that spread from the Southeast to create significant capacity issues across the IMS (IP multimedia Subsystem) core network that supports VoLTE calls.