Hadoop and Java get job scheduling support from Platform and Terracotta
Enterprise resource allocation tools updated
By Joab Jackson | Published: 11:49, 30 March 2011
Enterprise servers may get a lot busier, at least if their administrators adopt one of two freshly upgraded job schedulers, both of which promise greater control in the scheduling of workloads across many servers.
Platform Computing has outfitted its grid and cloud job scheduling tools to run distributed Hadoop MapReduce jobs across a network. And Terracotta has revamped its enterprise Java job scheduler with a new set of APIs (application programming interface) and a new GUI (graphical user interface).
To address the growing market for big data analysis, Platform Computing has outfitted both its Platform LSF (Load Sharing Facility) and Platform Symphony software packages with the ability to manage MapReduce distributed workloads. LSF offers long term scheduling for heavy workloads, while Platform Symphony is geared to more short term and near-real time scheduling.
Related Articles on Techworld
MapReduce, a component of Hadoop, can analyse data that is spread out across many nodes, with each node analysing its own data. As the popularity of MapReduce grows, organisations might find that they have multiple departments looking to run their own MapReduce jobs. The Platform software can provide a way to assure that access to the servers is spread out proportionally across all users.
While Hadoop has its own job scheduler, JobTracker, it lacks the nuanced controls of Platform's own schedulers, the company claims. Platform's software can run multiple jobs simultaneously, mix Hadoop and non-Hadoop jobs, and limit the number of nodes any one job can have. Users can run MapReduce across as many as 40,000 servers, the company claims.
While Platform Computing has concentrated on MapReduce scheduling, Terracotta has been updating its own job scheduler to better serve enterprise Java users.
Terracotta acquired the stewardship of the open source Quartz Scheduler when it purchased Quartz, the company, in November 2009. Companies such as Red Hat, VMware and Atlassian have embedded Quartz into their own products.
For the new version, the API of Quartz has been revamped to provide more insight and control over how servers are dedicated to jobs. A new management GUI has also been added, which will allow administrators a better view into which jobs are running and where they are being executed.
The commercial version of Quartz Scheduler allows users to designate specific machines to carry out jobs, or to apportion resources by RAM, CPU or specific operating systems.