Sybase embraces Google MapReduce

Who says there is a war raging between traditional SQL database vendors and the upstarts touting the so called 'NoSQL' technology?

A slew of relational database vendors, ranging from startups touting massively-parallel data warehousing appliances to grizzled veterans of the online transaction processing (OLTP) wars, are moving to support the MapReduce data processing technology invented by Google. Sybase is the latest vendor to jump on the NoSQL bandwagon. CTO Irfan Khan disclosed that the veteran database vendor is now working on adding MapReduce functionality inside its popular Sybase IQ analytic database.

The new functionality should help Sybase IQ crunch more data faster, says Khan. Sybase IQ is the most established of the column-based databases that have become trendy for BI and analytics applications. "By incorporating MapReduce in a user-defined function (UDF) approach, we'll be able to break down the complexity of a problem [and] push processing closer to where the data resides while keeping the friendly SQL-like exterior," Khan said.

Startups Aster Data and Greenplum kicked off the NoSQL trend last year by announcing plans to add the MapReduce technology to their respective data warehousing apps in order to accelerate the crunching of large analytical data sets. Proponents claim that MapReduce, along with its Java-based open source sibling, Hadoop, are faster than relational databases that hold huge amounts of data, think hundreds of terabytes or even petabytes, spread out over large scale server grids like Google's search server installations.

MapReduce and Hadoop usage began among web-centric companies like Yahoo, Facebook, and LinkedIn, all with data processing needs similar to Google's.

Meanwhile, SQL proponents argue that the advantages of MapReduce and Hadoop are limited only to certain types of data crunching, such as text indexing or data mining, and that the SQL language and tools are faster and better suited for wideranging needs of most enterprises.

Judging by the companies presenting at the first HadoopWorld conference, though, interest in the new technology is widespread. The list included speakers from Visa, JP Morgan Chase, Booz Allen, the New York Times and China Mobile.

Relational database pioneer Michael Stonebraker co-authored a paper earlier this year contending that that SQL technology still beats MapReduce in most cases. But that conclusion didn't stop Vertica Systems, the startup where he serves as CTO, from adding Hadoop functionality to its new Vertica 3.5 database.

Even the top three makers of traditional relational databases appear to be swayed by the shift in corporate interest from OLTP to analytics and BI.

IBM said in October that it has developed a Hadoop-based data analytics web service called M2, though it has not disclosed any plans to use the technology around its relational DB2 or Informix databases. Microsoft is reportedly working to add MapReduce-like features to the next version of its Flagship database, SQL Server 2008 R2 , codenamed Madison. Neither IBM or Microsoft officials immediately responded to a request for comment.

Meanwhile, Oracle said recently that it has already added MapReduce to its market leading database.

Analyst Curt Monash says he's most impressed so far by Aster Data's integration of SQL and MapReduce. "The guys at SAS [Institute, an Aster partner ], love them for it," he said.