Cray XK7

XK7 is scalable up to 500 cabinets, each contains 24 blades and each blade contains 4 nodes (1 CPU and 1 GPU per node).[2] The CPUs available are of the 16-core AMD Opteron 6200 Interlagos series and the GPUs are of the Nvidia Tesla K20 Kepler series.[2] Each CPU can be paired with either 16 or 32 GB of error-correcting code memory (ECC) while the GPUs have either 5 or 6 GB of ECC memory depending on the model of GPU used.[2] The nodes communicate with each other via the Gemini Interconnect; each Gemini chip services 2 nodes with a capacity of 160 GB/s.[2] Depending on the components used, a full cabinet will consume between 45 and 54.1 kW of electricity which is converted into heat; thus the cabinets need cooling, either by air or water.[2]

XK7 based machines run the Cray Linux Environment which incorporates SUSE Linux Enterprise Server.[2] Code to run on an XK7 machine can be written in a range of programming languages. The hybrid architecture requires different programming to conventional CPU-only supercomputers; Oak Ridge National Laboratory and the Swiss National Supercomputing Centre hold workshops to educate researchers on the new programming approach.[3][4]

The XK7 platform was announced on October 29, 2012 to coincide with the completion of Titan at Oak Ridge National Laboratory (ORNL).[5] Titan has 18,688 XK7 nodes, each containing an Opteron 6274 CPU with 32 GB of memory and a K20X GPU with 6 GB.[6] The computer has a theoretical peak performance of 27.1 petaFLOPS but in the LINPACK benchmark used by the TOP500 organisation to rank supercomputers it performed at 17.59 petaFLOPS, enough to take first place on the November 2012 list.[7][8] Titan uses 8.2 MW of electricity and is third on the Green500 list which ranks supercomputers by their energy efficiency.[9]

The National Center for Supercomputing Applications (NCSA) in Illinois has a machine, Blue Waters, using a combination of Cray XE6 and XK7 nodes. The machine has 3072 XK7 nodes and 22,752 XE6 nodes.[10] Each XE6 node has two Opteron 6276 and 32 GB of memory per CPU.[11] The XK7 nodes also have Opteron 6276 CPUs with 32 GB of memory and a K20X GPU with 6 GB.[11] Blue Waters has performed at over 1 petaFLOPS in benchmarks; however, the project managers do not believe in the relevance of the LINPACK benchmark used by the TOP500 organisation and therefore did not submit a benchmark test for ranking.[10]

The Swiss National Supercomputing Centre (CSCS) machine named Todi was upgraded to XK7 on October 22, 2012.[12] Todi has 272 nodes with Opteron 6272 CPUs with 32 GB of memory and a K20X GPU with 6 GB.[13][14] Todi has a theoretical peak performance of 393 teraFLOPS and performed at 274 teraFLOPS in the November 2012 TOP500 list taking 91st place.[14] Todi consumes 122 kW and is ranked fourth, one behind Titan, on the November 2012 Green500 list.[9][14]