Sun supercomputer takes on IBM's Blue Gene

Sun is aiming to wrest the world supercomputing crown from IBM's Blue Gene courtesy of a $59 million contract from the University of Texas for its Constellation design.

This is a 421 teraflop design, providing 421 million million floating point operations per second, potentially reaching 2 petaflops. Unfortunately IBM has dimmed Sun's system with a 3 petaflop version of its Blue Gene supercomputer. Sun could potentially hit the number two spot.

Sun's constellation

Constellation will be installed at TACC, the Texas Advanced Computing Centre, alongside other, lesser, supercomputers, and is a Linux cluster system, to be known as Ranger. It will have 3,288 nodes, starting out with 26,304 processing cores, using AMD's forthcoming Barcelona 4-core Opteron design, mounted on Sun blades. Ultimately there will be 1,302 Opterons providing 5,208 cores.

The initial memory will be 52.6TB with a final RAM capacity of 105TB. This will be backed up with 1.73PB of disk storage. The system components are connected by Infiniband with a huge, 3,456-port central switch designed by Sun co-founder Andy Bechtolstein. Its total bandwidth is 110TB/sec and it connects 1,152 cables with 12 connections per wire.

The benefit of a big switch is that inter-switch cables (needed if smaller switches were used) can be dispensed with, saving a lot of money as it's cheaper to build a big switch than link smaller switches. Six times fewer cables are needed in fact. Sun also says that the processors get a standard latency for data access this way.

It will need 3 megawatts of power to run, and a standard rack holds 768 cores.

Sun estimates that Constellation could scale to a 2 petaflop system with 1 exabyte of disk capacity - that's one million million million byes. It isn't enough to keep IBM at bay though.

The empire strikes back

IBM has announced Blue Gene/P, the second generation of the world’s currently most powerful supercomputer. Also a Linux cluster, it nearly triples the performance of its predecessor, Blue Gene/L, while remaining, according to IBM, the most energy-efficient and space-saving computing package ever built.

It is designed to run continuously at speeds faster than 1 petaflop - 1 quadrillion floating point operations per second - and could be configured to reach 3 petaflops (one quadrillion means one thousand million million).

This is rather more powerful than a laptop computer and IBM helpfully suggests that a home user would need a 1.5 mile high pile of laptops to have 1 petaflop equivalent computing power available.

Dave Turek, IBM's deep computing VP, said: "We see commercial interest in Blue Gene developing now in energy and finance, for example. This is on course with an adoption cycle - from government labs to leading enterprises – that we’ve seen before in the high-performance computing market.”

Green Blue Gene

IBM says Blue Gene/P is a green supercomputer, being at least seven times more energy efficient than any other supercomputer. Its design uses many small, low-power chips each connected through five specialised networks inside the system.

Blue Gene/L uses dual-core 700MHz chips. Blue Gene/P has four 850 MHz PowerPC 450 processors integrated on a single chip. Each chip is capable of 13.6 billion operations per second. A two-foot-by-two-foot board containing 32 of these chips churns out 435 billion operations every second, making it more powerful than a typical, 40-node cluster based on two-core commodity processors. Thirty-two of the compact boards comprise the 6-foot-high racks, meaning 4096 cores per rack - far more dense than Sun's core count per rack.

The one-petaflop Blue Gene/P configuration is a 294,912-processor, 72-rack system harnessed to a high-speed, optical - not Infiniband - network. A new SMP mode supports multi-threaded applications.

The U.S. Dept. of Energy’s Argonne National Laboratory, Argonne, Ill., will deploy the first Blue Gene/P in the U.S. beginning later this year.

In Germany, the Max Planck Society and Forschungszentrum Julich also plan to begin installing Blue Gene/P systems in late 2007. Additional Blue Gene/P rollouts are being planned by Stony Brook University and Brookhaven National Laboratory in Upton, N.Y., and the Science and Technology Facilities Council, Daresbury Laboratory in Cheshire, England.

Eat your heart out, Sun.