Follow Us

We use cookies to provide you with a better experience. If you continue to use this site, we'll assume you're happy with this. Alternatively, click here to find out how to manage these cookies

hide cookie message

Netflix releases customised Amazon Web Services control console

Netflix's home-built open source tool manages Amazon workloads

Article comments

Nefflix has released an open source console called Asgard that it developed to manage its deployments on Amazon Web Services.

A home-built console was necessary for Netflix, given the amount of work it conducts on Amazon's cloud services.

"For everyday large-scale operations, the AWS Management Console has not yet met the needs of the Netflix cloud usage model, so we built Asgard instead," said Joe Sondow, the Netflix software engineer for engineering tools who led development of the application.

Amazon's own management tools have a number of shortcomings, Sondow said. For instance, Amazon offers no way to automate the workflow around deploying an application. It does not provide a log of recent administrative actions. And it doesn't provide many ways to integrate in-house engineering tools, such as Jenkins.

Sondow built the application to address these issues, using the Grails framework. Named after the home of the Norse god of thunder and lightning, Asgard allows administrators to manage their Amazon cloud deployments from two perspectives unavailable on Amazon's own management interfaces, that of applications and clusters.

For Netflix, running an application may require multiple Amazon services, including those for load balancing, auto-scaling, launching and security. Asgard provides a single view for managing these services, organised around each application. Asgard maintains an application registry in Amazon SimpleDB, which keeps track of the many resources supporting a single application. It can also show how long each instance of the application has been running and when it was created.

Asgard has a similar interface for managing clusters, one that lists the individual nodes of each cluster. The software can remove individual nodes, a practice called rollback, if they prove to be troublesome. It also can manage a rolling push, in which all the nodes of a cluster are replaced one by one, without disrupting the operation of the cluster as a whole.

Netflix' use of Amazon is expansive. Except for billing and a number of other functions, "We run all of Netflix out of EC2," explained Jeremy Edberg, the lead cloud reliability engineer at Netflix, during a talk at the Qcon developer conference last week in New York City. The company had to use a public cloud because "we couldn't build data centres fast enough," he said. Netflix has 25 million customers who stream movies and televisions shows from the service.

To help its developers quickly create new features and services, Netflix has built an entire platform from CentOS and Ubuntu virtual servers running on Amazon. "We essentially built a PaaS," Edberg said. Developers can easily build Java or Python applications that will run automatically on the platform, which uses the Apache Tomcat, or another application server of the developer's choosing.

As for Asgard, "this a great tool that helps us be very quick and agile in our deployments and management of Amazon," Edberg said. "We are really hoping people will push patches back, because there is stuff that it doesn't yet do."

The code, available on GitHub, is available under a Apache License, Version 2.0.

Amazon itself appears to be pleased with Asgard. Amazon Chief Technology Officer Werner Vogels praised the technology in a short message posted on Twitter.



Share:

More from Techworld

More relevant IT news

Comments

intechnica said: I think its quite common for companies to feel that the AWS Management Console is lacking in some key areas from huge users of AWS like Netflix through to smaller companies and startupsIncreasingly people are coming up with home-built solutions to add in the functionality they need - for example we built a product called CloudFlex to simplify the process of starting stopping and backing up environments as well as creating templates for environments so its easier to go back to an old configuration laterWeve actually released CloudFlex as a product for external users - I wonder whether Netflixs Asgard will ever see action outside of Netflix one day



Send to a friend

Email this article to a friend or colleague:

PLEASE NOTE: Your name is used only to let the recipient know who sent the story, and in case of transmission error. Both your name and the recipient's name and address will not be used for any other purpose.

Techworld White Papers

Choose – and Choose Wisely – the Right MSP for Your SMB

End users need a technology partner that provides transparency, enables productivity, delivers...

Download Whitepaper

10 Effective Habits of Indispensable IT Departments

It’s no secret that responsibilities are growing while budgets continue to shrink. Download this...

Download Whitepaper

Gartner Magic Quadrant for Enterprise Information Archiving

Enterprise information archiving is contributing to organisational needs for e-discovery and...

Download Whitepaper

Advancing the state of virtualised backups

Dell Software’s vRanger is a veteran of the virtualisation specific backup market. It was the...

Download Whitepaper

Techworld UK - Technology - Business

Innovation, productivity, agility and profit

Watch this on demand webinar which explores IT innovation, managed print services and business agility.

Techworld Mobile Site

Access Techworld's content on the move

Get the latest news, product reviews and downloads on your mobile device with Techworld's mobile site.

Find out more...

From Wow to How : Making mobile and cloud work for you

On demand Biztech Briefing - Learn how to effectively deliver mobile work styles and cloud services together.

Watch now...

Site Map

* *