National Computational Infrastructure

NCI

Detailed System Configuration

The following provides some additional detail about the system configuration:

  • 96 nodes in 24 system boards, each containing:
    • One 16-core 1.848GHz Fujitsu SPARC64 IXfx cpu
      • L1 cache (on chip): 32KB/core (I) + 32KB/core (D)
      • L2 cache (on chip): 12MB per 16-core cpu
  • 32 Gbytes DDR3-1333 memory
  • Total of 3TB of RAM on compute nodes
  • 4 dual socket, 6-core Fujitsu RX200 S7 servers for FEFS file service
  • Approximately 122 TBytes of usable global storage using 4 disk trays, each with 10/12 x 3TB SAS drives
  • One Mellanox Infiniband switch with 36 FDR IB ports for both MPI and FEFS filesystem traffic
Compute Nodes
Model Fujitsu PRIMEHPC FX10 node

http://www.fujitsu.com/global/services/solutions/tc/hpc/products/primehpc/

Housing Four-node system board (Fujitsu proprietary rack unit)
processor 1 x Fujitsu SPARC64 IXfx (16-core, 1.848GHz)

http://www.fujitsu.com/downloads/TC/sc11/sparc64-ixfx-sc11.pdf

Memory 4GB DDR3 1333Mhz

3TB total

96 x 32GB nodes

Disk 2TB 7200rpm SAS

 

Interconnect
Interconnect
(compute nodes & I/O nodes)
6D mesh/Torus – “Tofu” interconnect
Interconnect (filesystem) FDR Infiniband
IB switch 1 x 36-port Mellanox SX6036 switch

http://www.mellanox.com/page/products_dyn?product_family=132&menu_section=49

IB cables 8

 

Storage
Storage Fujitsu Eternus DX80 S2 Hardware RAID 6

http://www.fujitsu.com/fts/products/computing/storage/disk/eternus/dx80/

Scalable unit 2 OSS, 2 Eternus DX80 S2, 44 disks
OST Disks 3TB SAS
# OST Disks 44
Usable TB 122
Raw TB 144
# OSS 2
OSS memory 16GB
IO Bandwidth 4GB/s
MDT disks 11

In Collaboration With