Measuring Broadband Performance

On July 19, the Federal Communications Commission released “Measuring Broadband America”, a report, the FCC notes, revealing that broadband providers have significantly improved accuracy in actual versus advertised broadband speeds during the past year and that consumers are subscribing to faster speed tiers and receiving faster speeds than ever before. The FCC surveyed more than 5,500 users in April. Overall, the 13 top US broadband providers, representing four-fifths of all U.S. landline broadband connections, are coming much closer to consistently delivering their advertised speeds.

The report has plenty of great information, writes GigaOm’s Stacey Higginbotham, such as the quality of service broken down by speed tiers as well as a chart indicating that as you give people faster speeds they tend to consume more data. That may be obvious, but it’s always nice to have numbers that can back this information up. And it’s even nicer to see areas such as advertised speed improve over the previous year when the FCC began collecting the data. Shining light on broadband performance does improve it.

The FCC highlights two key areas of improvement:

• First, broadband providers’ promises of performance are more accurate. In the time period measured for the first Measuring Broadband America Report in August 2011, the average broadband provider delivered 87 percent of advertised download speed during times when bandwidth demand was at its peak. During the time period measured for the July 2012 Report, that number rose to 96 percent. FCC analysis indicates that the improvements of Internet Service Providers (ISPs) in meeting their advertised speeds were largely driven by improvements in network performance, and not downward adjustments to the speed tiers offered.
• Second, consumers of broadband providers covered by the Report are subscribing to faster speed tiers and receiving faster speeds than ever before. The FCC found that the average speed tier that consumers were subscribing to increased from 11.1 Megabits per second (Mbps) to 14.3 Mbps, an almost 30 percent increase in just one year. Because providers also did a better job in the testing period for this report of meeting or exceeding their advertised speeds, the actual increase in experienced speeds by consumers was even greater than the advertised speed, from 10.6 Mbps to 14.6 Mbps, representing an almost 38 percent improvement over the one year period.

The report also finds:

• Verizon's fiber network, Cablevision, Comcast Corp, Mediacom Communications and Charter Communications Inc all routinely delivered nearly 100 percent or greater of the speed advertised, even during the hours of highest demand;
• Frontier performs the worst with speeds just 79 percent as fast as advertised. Qwest at 83 percent and AT&T at 87 percent also fall short. Verizon's DSL service is only 87 percent as fast as the company advertises;
• Cable-based services delivered 99 percent of advertised speeds, and fiber-to-the-home services delivered 117 percent of advertised speeds;
• Average peak period download speeds varied from a high of 120 percent of advertised speed to a low of 77 percent of advertised speed. This is a dramatic improvement from last year where these numbers ranged from a high of 114 percent to a low of 54 percent.
• On average, ISPs had a 3 percent decrease in delivered versus advertised download speed between their 24 hour average and their peak period average.
• With the exception of one provider, upload speeds during peak periods were 95 percent or better of advertised speeds. On average, across all ISPs, upload speed was 107 percent of advertised speed.
• On average, fiber-to-the-home services delivered 106 percent, DSL-based services delivered 103 percent, and cable-based services delivered 110 percent of advertised upload speeds.
• Average upload speeds among ISPs ranged from a low of 91 percent of advertised speed to a high of 122 percent of advertised speed.
• The success rates hold up for latency, which measures the time it takes for data to move between points on a network. High latency times, typically measured in milliseconds, can cause hiccups in video streams or voice transmissions. Fiber scored best in this category, followed by cable and DSL. Overall, however, latency rates did not improve. The report suggests that, "the primary causes of latency are intrinsic to the service architecture and not amenable to significant improvement."

“The strong improvements we see in ISP performance from the first test to today," said FCC Chairman Julius Genachowski, "is a case study in how smart, government-driven transparency drives better outcomes. There is no doubt that information from our report last year fueled competition in the marketplace. High-performers touted their rankings in TV and radio ads, on blogs, in articles—the word got out far and wide. And low performers changed their practices and speeds.”

FCC Commissioner Jessica Rosenworcel said that the tests conducted for the report “do more than shine a light on actual speeds delivered to consumers. They remind us that it is essential that consumers fully understand these speeds and how they can use the capacity they provide. So going forward, I welcome expanding this measurement project to new technologies and looking for new ways to provide consumers with the information they need and the broadband services they want.”

Commissioner Robert McDowell pointed out that the survey did not include wireless broadband and said that needed to be made clear to consumers and rectified. Chairman Genachowski said the FCC was working on it and agreed that mobile broadband info was needed as well. Commissioner McDowell said the report was an encouraging sign of a competitive marketplace and that it was important not to do anything to discourage the private capital investments that drive that competition.

Commissioner Ajit Pai said that they key question was whether consumers were getting what they had paid for, and the answer was a resounding "yes."

Timed to coincide with the FCC's report is The Cost of Connectivity report from the New America Foundation, which also assists the FCC on its broadband report. That report finds cities with municipal-created broadband systems such as Bristol (VA), Chattanooga (TN), and Lafayette (LA) are among the fastest in the US. Those cities, as well as San Francisco, offer good models for local governments looking to build their own broadband systems or foster local ISP growth, the report finds. However, many state governments are stifling such possibilities with legislation supported by existing broadband providers looking to protect their own interests, says Benjamin Lennett of New America's Open Technology Institute. "We have to figure out a way to increase the level of competition or we are all going to pay through the nose, and speeds are not going to keep up."

Back in April, the Benton Foundation and the Institute for Local Self-Reliance released Broadband At the Speed of Light: How Three Communities Built Next-Generation Networks. Frustrated by ever-increasing prices for telecommunication services and the reluctance of incumbent providers to upgrade their networks to meet 21st century needs, more than 150 communities have built their own citywide cable and fiber-to-the-household (FTTH) networks. Against great odds and in the face of ferocious opposition by the existing telephone and cable companies in the courts, at the legislature, and in the marketplace, the vast majority have succeeded.

To understand how this has occurred and to extract lessons that might be useful for cities deciding whether to build their own networks, we undertook an in-depth examination of three municipally owned networks in Bristol, Chattanooga, and Lafayette.

Each of these communities already had access to the Internet via DSL and cable. But in the words of Lafayette City-Parish President Joey Durel, “They wanted more.” Without investment in next-generation networks, these cities feared they would be left behind in the transition to the digital economy of the Internet era.

In each of these cases, the local public power utility took the lead in creating the new network—a characteristic of nearly every citywide publicly owned community fiber network in America. Each community had to navigate difficult seas, buffeted by lawsuits that dragged out construction schedules, state legislation that imposed additional burdens on public networks, and huge corporate competitors benefiting from a multitude of scale advantages. In each of these cases, the communities found their network to be a major economic development asset, generating or preserving hundreds of well paying jobs.

The New America Foundation was also a lead organizer on a letter from over 60 preeminent researchers for network and Internet measurement sent to Chairman Genachowski this week urging the FCC to commit to the following principles and actions for broadband measurement. Open data and an independent, transparent measurement framework, the researchers argue, must be the cornerstones of any scientifically credible broadband Internet access measurement program. The researchers are advocating for the FCC to adopt principles of openness and transparency and to allow the scientific process to serve as the foundation of the broadband measurement program.

1. All network data collected should be released publicly, in its raw, non-aggregated form, no later than one year after its collection date.
2. The tests used to measure the network should be open-source, allowing the methodology used in measurement to be vetted and improved by collective scientific insight.
3. The measurement infrastructure on which the tests run should be openly documented, consistently run, and independently managed by the research community.
4. The analytic and statistical methodologies applied to the raw data in the production of a published report should be released openly, allowing peer review and replication of results by independent researchers.
5. The FCC should make commitments to operational openness that explicitly includes participants from the research and public interest community.

New America’s Sascha Meinrath, Google’s Vint Cerf, and a large body of network researchers founded Measurement-Lab (M-Lab) which promotes scientifically sound broadband measurement at scale. M-Lab provides a broadly distributed, independently managed, open and consistent server platform on which researchers and participating partners run broadband measurement tools. As a condition for access to the platform, researchers and participating partners must make all data collected by these tools publicly available for the benefit of research. This requirement allows research at scale, and allows published results to be vetted and confirmed by the scientific community.

As a part of this commitment to sound science and transparent measurement, M-Lab has worked with the FCC on its broadband measurement program. M-Lab provides use of its platform to the collaborative free of charge, and UK company SamKnows runs its test on M-Lab's servers.

Thus far, all of the data collected and used in the FCC's published reports has been collected on M-Lab's platform. Because this program produces the canonical US statistics on broadband performance, we have been gratified to contribute, and have lauded the FCC's commitment to openness and transparency.

But sound science requires openness and transparency to ensure verifiable and replicable results. In recent months M-Lab has expressed concern that the FCC’s measurement program was drifting away from its commitments to open data and transparency. In particular, a proposal to include ISP-run servers in the measurement process raised alarm. This change would effectively transform an open program into a closed program, giving the ISPs control over measurements purported to gauge their performance.

Benton will continue to watch for any relay any researchers' concerns that changes to the FCC's broadband measurement program that could put at risk "government-driven transparency [that] drives better outcomes." 'Til then, we'll see you in the Headlines.

Notes:
The ISPs in the FCC study, ranked by performance as a percentage of advertised speed, were: Verizon's fiber network (120 percent), Cablevision (120 percent), Comcast (103 percent), Mediacom (100 percent), Charter (98 percent), Time Warner Cable Inc (96 percent), Cox Communications (95 percent), Insight Communications Co (92 percent), CenturyLink Inc (89 percent), AT&T Inc (87 percent), Verizon's DSL network (87 percent), Windstream Corp (84 percent), Qwest Communications (83 percent) and Frontier (79 percent).
back to top