Data creation outstrips storage for first time

For the first time, the amount of digital information created each year has exceeded the world's available storage space, according to a new IDC report.

"This is our first time ... where we couldn't store all the information we create even if we wanted to," states the report, titled The Diverse and Exploding Digital Universe.

The amount of information created, captured and replicated in 2007 was 281EB (exabytes or 281 billion GB). This is 10 percent more than IDC previously believed - and more than the 264EB of available storage on hard drives, tapes, CDs, DVDs and memory.

IDC revised its estimate upward after realising it had underestimated shipments of cameras and digital TVs, as well as the amount of information replication.

The 2007 total is well above that of 2006, when 161EB of digital information was created.

We're not actually running out of storage space, IDC notes, because a lot of digital information doesn't need to be stored, such as radio and TV broadcasts consumers listen to and watch but don't record, voice call packets that aren't needed when a call is over, and surveillance video that isn't saved.

But the gap between available storage and digital information will only grow, making it that much harder for vendors and enterprises to efficiently store information that is needed.

In 2011 there will be nearly 1,800EB of information created, twice the amount of available storage, IDC predicts. One long-term experiment planned for the soon-to-open Large Hadron Collider in Switzerland by itself will create an amazing 300EB of data per year.

EMC's president of content management, Mark Lewis, doesn't think we'll ever hit the point where the world's available storage is exceeded by the amount of information we need to store. "With the price points of storage continuing to decline, I don't think we're ever going to create some kind of storage shortage," he says.

Enterprises and their employees create about a third of new data, but enterprises are ultimately responsible for maintaining the security, privacy and reliability of 85 percent of all data, according to IDC.

Information growth is placing greater importance on retaining data in lower-cost, environmentally sound ways, with lower-performance drives, archiving and powering down storage devices containing rarely accessed data, Lewis says.

About 70 percent of new information is created when individuals snap pictures, make VoIP calls, upload content to YouTube and send emails. But more than half of the information related to individuals isn't directly created by them.

Rather, the bulk of this digital content is a person's "digital shadow," information about individual human beings sitting in cyberspace. Digital surveillance photos, web search histories, banking and medical records and general backup data all contribute to your digital shadow.

EMC (who sponsored the report) have teamed up with IDC to develop a calculator for you to measure your own digital footprint. The website also has a running tally of information created and replicated so far this year