Californian uni offers open source-tool storage management tool
Dealing with petabytes of data.
By Network World staff | Published: 06:09, 12 February 2008
The supercomputing group from the University of California at San Diego has upgraded its data management software and has made it available as an open source offering for handling petabytes of data.
The Data-Intensive Computing Environments (DICE) group at the San Diego Supercomputer Center (SDSC) has issued iRODS (Integrated Rule-Oriented Data System) 1.0, which the outfit says improves on and goes beyond the Storage Resource Broker it has developed over the past 10 years.
"iRODS equips users to handle the full range of distributed-data-management needs, from extracting descriptive metadata and managing their data to moving it efficiently, sharing data securely with collaborators, publishing it in digital libraries, and finally archiving data for long-term preservation," said Reagan Moore, director of the DICE group.
The heart of iRODS is a rules engine that supports customised data-management needs, such as retaining records in one place for a set period of time, then automatically destroying them.
While iRODS is simple enough to be used by an individual user, at the SDSC it is used to handle a petabyte of data and 200 million files for 5,000 users, according to the university. The US National Archives and Records Administration (which funded the project along with the National Science Foundation) and Ocean Observatories Initiative are among other iRODS users.
Version 1.0 of iRODS runs on Linux, Solaris, Macintosh and AIX, with Windows support on the way. It runs on the open source PostgreSQL database and Oracle databases, and works across multiple physical servers.
