NASA builds open source Nebula cloud platform

The Project: Develop the federal government's first open source cloud computing platform to provision high performance computing, storage and network services for NASA's research community and enable scientists and engineers to share large, complex data sets with partners and the public.

The Business Case: Traditionally, it took several months and hundreds of man-hours to procure, configure and maintain new IT infrastructure for NASA scientists and engineers. Computing power dedicated to a single project or research area was underused, and energy costs were skyrocketing. IT leaders saw cloud computing as a potential solution to these technology challenges.

First Steps: NASA's cloud services project, nicknamed Nebula, was hatched two years ago. It transformed shipping containers into modular data centres, using virtualisation to support up to 16 petabytes of data each and capable of halving energy use. For more than a year, Nebula was something "we were working on in the background," explains Ames Research Centre CIO James Williams. For example, the Marshall Space Flight Centre, another NASA agency, tested Nebula for predictive weather modelling, a project that would normally take a month to set up. Instead, Williams says, it was up and running in four days.

Nebula rocketed to the fore last autumn, when federal CIO Vivek Kundra asked Ames to host USASpending.gov, a searchable database of federal funding. NASA launched an agencywide cloud computing initiative in April, and in July, Nebula's core software was chosen for OpenStack, an open source initiative aimed at driving cloud computing standards. OpenStack has hundreds of developers working on the code for any given project. "It could save taxpayers millions of dollars," says Williams.

Now users throughout NASA can unilaterally provision and manage IT resources for low security applications on demand. Next year, IT will launch Nebula's platform as a service: a shared development framework, code repository and set of Web services that developers can use to deploy secure, policy-compliant software-as-a-service applications.

NASA has spent more than $10 million on the project, but it will be another year before the return is clear. "We want to get to a day where for every dollar you invest in this, you save six," says Adrian Gardner, CIO with Goddard Space Flight Centre, a NASA division that provides staff and infrastructure for Nebula. "We're not looking to shift everything to the cloud. It's just another tool."

What to Watch out For: Training technologists isn't enough. "We're known for doing things most people would characterise as being impossible. So culturally, the cloud is a fit," Gardner says. But convincing researchers accustomed to working with several computers under their desks takes time. IT runs workshops to demonstrate the new processes for provisioning systems and promotes Nebula to individual scientists.

NASA had no open source policy (though one is now in the works), which delayed the agency joining OpenStack and slowed Nebula's progress. Security is another concern. Before moving sensitive data to Nebula, NASA is getting advice from other agencies that handle such information.