Apache refines text search stack Lucene and Solr

Further refining an already widely used full-text search engine, the Apache Software Foundation has updated Lucene to execute searches more quickly across multiple servers, the development team announced. It has also improved performance of Lucene's accompanying search platform, Solr.

While not a radical upgrade, the new 3.3 versions of Solr and Lucene do include numerous improvements, said Alan Pelz-Sharpe, a principal analyst and director of the content management system analysis firm Real Story Group, which tracks the progress of these technologies.

Created by Doug Cutting, who also originated Hadoop, Lucene is a Java full-text distributed search engine, one aimed to run speedily across multiple platforms and across multiple servers. Solr provides the interface from which searches can be issued and results presented back to the user.

Lucene and Solr are widely used as the basis for Intranet and internal enterprise search engines, where they can be used to index and retrieve word documents, PDFs, databases, geo-spatial content and other forms of rich media content. Enterprise software system providers such as IBM bundle Lucene in their own offerings, Pelz-Sharpe said.

Lucene is a "mature and well-proven" text engine, Pelz-Sharpe said. Because it is open source, organisations can fine tune how the technology works, though its use does require a fair amount of expertise, he warned.

With the new release of Lucene, its ability to merge its own indexes has been improved. Lucene has had the ability to merge multiple indexes under a single index to save time, but there is now a new method to control how aggressively it merges. The software can now also shard the indexes, or split them, across multiple servers, which could cut search time even further.

With Solr, the project team had reduced the amount of working memory, or RAM, that the software requires for some functions by an order of magnitude. Also, search results can now be grouped by field or query.

Both Lucene and Solr can be downloaded at no cost from Apache or mirror servers.