National Computational Infrastructure

NCI

Detailed System Configuration

Summary

Raijin (Fujitsu Primergy)

Nodes

3,609

Cores

57,864

kSFRB

1,598

SFRB/core

27.8

SSPsys

6.377

SSPapp

6.612

SSP

6.493

Peak Tflops

1,195

Notes:

1         SFRB is an abbreviation for the SPECfp_Rate_Base2006 performance metric.

2         SSP refers to the sustained system performance metric, a performance comparator that relates the aggregate performance of Raijin over its antecedent, Vayu. This takes into account the relative size of the two systems, and the relative performance of the benchmark suite on the two systems to generate a single metric, which is the factor by which the performance of the new system is superior to that of its antecedent. It is the geometric mean of SSPsys   (determined from the SPECfp test), and SSPapp  (derived from the geometric mean of a sequence of application benchmark tests that are representative of the system workload).

Detailed below are the specifications of Raijin’s compute nodes, interconnect, and storage

Compute Nodes
Model CX250 nodes in CX400 chassis

http://www.fujitsu.com/fts/products/computing/servers/primergy/scale-out/cx400/

http://www.fujitsu.com/fts/products/computing/servers/primergy/scale-out/cx250/index.html

Housing Four-node 2U unit
Processor 2 x Intel Sandy Bridge E5 2670 (8-core, 2.6GHz)

http://ark.intel.com/products/64595/Intel-Xeon-Processor-E5-2670-%2820M-Cache-2_60-GHz-8_00-GTs-Intel-QPI%29

Memory 8GB DDR3 1600Mhz

158TB total

2395 x 32GB nodes

1125 x 64GB nodes

72 x 128GB nodes

Disk 500GB 7200rpm SATA
PSU 2 x 1400W (non-redundant)

per four-node CX400

 

Interconnect
Interconnect Non-blocking fat-tree Mellanox FDR Infiniband
Core switches 6 x 648 port

http://mellanox.com/content/pages.php?pg=products_dyn&product_family=122&menu_section=49

Leaf switches

 

205 for compute + 5 for servers

4 x 36-port leaf switches

at the top of each compute rack

72 copper IB cables within each rack

http://mellanox.com/content/pages.php?pg=products_dyn&product_family=132&menu_section=49

IB cables 7384

 

Storage
Storage DDN SFA12000 Hardware RAID 5 couplets each with 840 disks

http://www.ddn.com/products/sfa12k

Scalable unit 10 OSS, 1 SFA12000, 840 disks
OST Disks 3TB SAS
# OST Disks 4200
Usable PB 9.36
Raw PB 12.6
# OSS 50
OSS memory 128GB
IO Bandwidth 120GB/s
MDT disks 192 (48 quad mirrors)

In Collaboration With