Follow Us

We use cookies to provide you with a better experience. If you continue to use this site, we'll assume you're happy with this. Alternatively, click here to find out how to manage these cookies

hide cookie message

Facebook goes open source with its workhorse embedded data store

Facebook's RocksDB is an embeddable, persistent key-value store designed for fast reads and writes

Article comments

Continuing its practice of sharing internally developed software, Facebook has released as open source RocksDB, the embedded data store the company developed to serve content to its 1.2 billion users.

The company has posted the code for the database on Github, in hopes that others, both in industry and the academic community, will refine the software. With Facebook's emphasis on scalability, RocksDB may be of interest to other Internet services and enterprises that are building high-traffic apps for customers and employees.

In its newfound liberation, RocksDB joins other software that Facebook has released as open source. Facebook has posted the source code this year for the HipHop PHP virtual machine, the Presto query engine, the Flashcache caching software, and the Corona Hadoop scheduler, all of which were developed in-house.

RocksDB is not a full database of either a SQL or NoSQL variety. It has no indexing capabilities nor can it parse SQL queries. The software is a persistent key-value store designed to quickly serve material to users, according to a blog post by Dhruba Borthakur, an engineer on Facebook's database engineering team. It can also write or delete material to a database, but offers no advanced parsing capabilities.

Written in C++ as a library, RocksDB can be embedded into other applications, particularly those that need access to large sets of data with very low latency, such as a spam detection application or a search engine.

RocksDB is actually a fork of Google's LevelDB, a simple non-SQL data store for reading, writing and deleting data. Facebook, however, found that LevelDB did not perform well with data sets that could not fit into the server's working memory, so engineers modified Google's open-source code.

Facebook also modified LevelDB so that it can be run across many processor cores of a server. Because of this work, it can support extremely fast I/O: Facebook tests showed that the data store can perform 10 times faster for random writes, as well as 30 percent faster for random reads over LevelDB.

Borthakur offered a few details of how Facebook uses RocksDB in production. In one configuration, the data store is run in front of 10 solid-state drives, striped to support a million reads and writes a second. The software now manages over a petabyte of data that it regularly serves to Facebook's users.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com



Share:

More from Techworld

More relevant IT news

Comments



Send to a friend

Email this article to a friend or colleague:

PLEASE NOTE: Your name is used only to let the recipient know who sent the story, and in case of transmission error. Both your name and the recipient's name and address will not be used for any other purpose.

Techworld White Papers

Choose – and Choose Wisely – the Right MSP for Your SMB

End users need a technology partner that provides transparency, enables productivity, delivers...

Download Whitepaper

10 Effective Habits of Indispensable IT Departments

It’s no secret that responsibilities are growing while budgets continue to shrink. Download this...

Download Whitepaper

Gartner Magic Quadrant for Enterprise Information Archiving

Enterprise information archiving is contributing to organisational needs for e-discovery and...

Download Whitepaper

Advancing the state of virtualised backups

Dell Software’s vRanger is a veteran of the virtualisation specific backup market. It was the...

Download Whitepaper

Techworld UK - Technology - Business

Innovation, productivity, agility and profit

Watch this on demand webinar which explores IT innovation, managed print services and business agility.

Techworld Mobile Site

Access Techworld's content on the move

Get the latest news, product reviews and downloads on your mobile device with Techworld's mobile site.

Find out more...

From Wow to How : Making mobile and cloud work for you

On demand Biztech Briefing - Learn how to effectively deliver mobile work styles and cloud services together.

Watch now...

Site Map

* *