Follow Us

We use cookies to provide you with a better experience. If you continue to use this site, we'll assume you're happy with this. Alternatively, click here to find out how to manage these cookies

hide cookie message

VMware makes Hadoop enterprise-friendly with Project Serengeti

Making Project Serengeti available free through Apache continues a trend by VMware to embrace open standards

Article comments

VMware announced "Project Serengeti" on Wednesday, which is open source code that optimises Hadoop for use in VMware virtualised environments.

Bringing cloud-like benefits to the leading big data analytics tool will make it faster and easier to deploy and manage a variety of Hadoop distributions on VMware machines, company officials say.

"VMware has been working on cloud computing and virtualisation for quite some time, and big data is one of the hottest trends in IT. Now, we're bringing those worlds together," says Fausto Ibarra, senior director of product management for VMware. "With these announcements, Hadoop can become a first-class client in IT infrastructures."

Making Project Serengeti available free through Apache also continues a trend by VMware to embrace open standards. Its platform-as-as-service (PaaS) offering, Cloud Foundry, for example, is also open source. Ibarra says VMware wants Project Serengeti to be widely adopted within the Hadoop community and compatible with all the various Hadoop distributions, so open source was the way to go.

Project Serengeti is an important move to make Hadoop enterprise-friendly, says Tony Baer, an analyst at Ovum. "This will help Hadoop become more mainstream," he says. There are a variety of use cases where Hadoop could benefit from running in a virtualised environment, such as if an enterprise wants to experiment with a new feature on a dataset, but not expose the entire cluster.

Hadoop still in early stages of adoption by big enterprises

Hadoop, which is an open source software framework for managing massive amounts of unstructured data, is used by some of the top IT shops in the world, such as Yahoo and Facebook, but is still in its early stages of adoption across most mid- to large-size enterprises. Experts say VMware's announcement today, along with other Hadoop-related news this week, further legitimises the Hadoop market and could spur more companies to begin exploring the potential value of big data analytics.

With Project Serengeti, VMware has optimised Hadoop to run on virtualised infrastructure, compared to physical servers which do not run a hypervisor. While Hadoop clusters are currently running on virtualised machines in some instances now, VMware says supporting Hadoop clusters on its series of market-leading virtualisation products will open Hadoop up to be more easily deployed in enterprise settings. Using virtualised servers allows additional virtual machines to be deployed quickly and scale elastically while ensuring high availability and optimal hardware utilisation, the company says.

Hadoop support is initially programmed to run on VMware vSphere virtualisation products and it is compatible with Hadoop distributions such as those from Cloudera, MapR, IBM and Greenplum. Ibarra says there will be continued advancement of Project Serengeti to extend support to new Hadoop distributions and feature sets

More skilled workers need to manage Hadoop clusters

Ibarra says VMware officials have seen three major use cases for Hadoop among customers: One is in companies that are testing the platform and have less than 20 nodes or so. These customers, he says, are ideal for virtualised distributions of Hadoop because it will not require large new capital expenses if Hadoop can run on legacy vShphere private clouds.

A second customer set has an expanded use of Hadoop, up to 100 nodes or so, Ibarra says, and may be looking to take advantage of the dynamic elasticity Project Serengeti allows Haddop to leverage. A third use case is for the early Hadoop adopters, who are running hundreds of nodes and are looking for advanced uses. Almost any business today, he says, will find some use for Hadoop given the vast amounts of unstructured data produced through web traffic that can be analysed.

Carl Brooks, a cloud analyst at the 451 Research Group, says VMware is not the first to run Hadoop on virtualised machines, so the more significant news is that more vendors are recognising Hadoop's importance and potential, and are offering services around it. HortonWorks, for example, announced on Tuesday a Hadoop distribution compatible with VMware vSphere.

Hadoop is still early on its enterprise adoption phase though, says Ovum's Baer. A lack of skilled workers to manage Hadoop clusters and interpret the data Hadoop creates is another challenge for curious enterprises, he says.


More from Techworld

More relevant IT news


PhillRoberts said: Kind of curious as to how effective mapreduce jobs would be in a virtual environment Doesnt the very nature of it make bare metal a requirement for it to be performant

Send to a friend

Email this article to a friend or colleague:

PLEASE NOTE: Your name is used only to let the recipient know who sent the story, and in case of transmission error. Both your name and the recipient's name and address will not be used for any other purpose.

Techworld White Papers

Choose – and Choose Wisely – the Right MSP for Your SMB

End users need a technology partner that provides transparency, enables productivity, delivers...

Download Whitepaper

10 Effective Habits of Indispensable IT Departments

It’s no secret that responsibilities are growing while budgets continue to shrink. Download this...

Download Whitepaper

Gartner Magic Quadrant for Enterprise Information Archiving

Enterprise information archiving is contributing to organisational needs for e-discovery and...

Download Whitepaper

Advancing the state of virtualised backups

Dell Software’s vRanger is a veteran of the virtualisation specific backup market. It was the...

Download Whitepaper

Techworld UK - Technology - Business

Innovation, productivity, agility and profit

Watch this on demand webinar which explores IT innovation, managed print services and business agility.

Techworld Mobile Site

Access Techworld's content on the move

Get the latest news, product reviews and downloads on your mobile device with Techworld's mobile site.

Find out more...

From Wow to How : Making mobile and cloud work for you

On demand Biztech Briefing - Learn how to effectively deliver mobile work styles and cloud services together.

Watch now...

Site Map

* *