Data centre consolidation runs rampant

While it used to be common for every enterprise to have its own data centre for delivering and receiving web traffic, a new study from security vendor Arbor Networks suggests that this is no longer the case.

In its Internet Observatory report, Arbor notes that consolidation of content providers has led to the rise of  "a small number of very large hosting, cloud and content providers" that generate and consume an estimated 30 percent of all Internet traffic. For instance, Arbor estimates that Google alone accounts for six percent of all Internet traffic in the world.

Arbor chief scientist Craig Labovitz said that there are several reasons for this migration of traffic from individual enterprise data centres to "hyper giants" such as Google, Facebook and Microsoft, including the rising costs and recourse demands of maintaining a data centre and the aggressive efforts by large companies to buy up video, email and other web service companies.

Additionally, he says companies that built their own data centres years ago found them quickly outdated and that they didn't have the money to properly upgrade them.

"Until a few years ago, there had been an overabundance of data centres," he says. "The data centres built five years ago are now out-of-date and there are entire generations of data centres where there's no way to upgrade them."

The answer for companies, he says, has been to consolidate their infrastructure through virtualisation or to outsource many of their IT operations to the cloud.

"Starting with outsourced web email, we have seen a large migration of web traffic out of small enterprise data centres and toward these large players," Labovitz explains. "The cost of data centres had started to affect companies' bottom lines and that has set the stage for what's starting to happen in the transit market."

Arbor says that another consequence of content providers and content delivery networks becoming larger has been the decreasing importance to Tier-1 transit providers such as Verizon Business, AT&T and Level 3 in delivering web traffic. And because these companies have lost some of their profitability in the transit market, Arbor says that they've turned themselves more toward value-added services.

"Over time, IP connectivity services became indistinguishable from one provider to the next," says Arbor in its research brief.

"In response, providers started competing chiefly on price. This price competition drove down IP wholesale market prices and forced many Tier 1 networks to pursue higher-value product offerings such as CDN, cloud computing and a greater focus on private enterprise offerings."

Arbor conducted its study of global traffic patterns by analysing nearly 3,000 peering touers across nine Tier-1, 48 Tier-2 and 33 consumer and content providers in four different continents. Arbor said that at its peak, "the study monitored more than 12 terabits-per-second of offered load and a total of more than 256 exabytes of Internet traffic."