Standards group looks to set SOA benchmark

A standards body is looking to address one of the major problems with SOA, that of measuring performance. The Standard Performance Evaluation Corporation. (SPEC) has formed a working group that will examine ways to measure performance for middleware and database applications within an SOA framework.

The group is looking to develop an initial benchmark based around three parts of a typical SOA deployment infrastructure:

• the services that sit on top of application servers using web services;
• Enterprise Service Bus (ESB) technologies that connect and mediate the services; and
• choreographing services into larger composite applications through BPEL (Business Process Execution Language) technologies.

Andrew Spyker, chair of the SPEC SOA Performance Benchmark group, said that the challenge was to develop a benchmark that would appeal to a wide range of vendors, each with different interests. "SPEC is the ideal organisation for tackling this challenge. We've faced similar challenges in developing worldwide standards for performance measurement in areas such as CPU, graphics/workstations, web and file servers and Java applications"

He said that the important thing was to start with what the working group believed to be the most important elements to benchmark." We will then introduce additional technologies as needed in the future. In all of our work, we'll look to measure performance in ways that are typical of customer deployments, while being flexible enough to cover current vendor implementations."

To that end, SPEC is looking for input from enterprise architects and IT managers to ensure that the working group understands customer needs for developing the appropriate benchmarks.

Spyker said that it was important that the benchmark was developed in such a way that it could satisfy business and IT users alike. "Specifically, the benchmark needs allow for the ability to change implementation quickly - providing business users with quicker turnaround time on changes to IT as the business conditions change. The implementation should also allow business services to be re-used in ways that make sense at a business level."