National Computational Infrastructure

NCI

Lustre File System (/g/data)

Description: The global Lustre file systems is designed to be a high speed file system that is accessible to the major systems operated by NCI.  The current system, which is presently under redevelopment, is 900 TB in size, and can be accessed via Infiniband at 5GB per second, or via NFS over Gigabit Ethernet. 

At present there is no automated back up in place for this file system. Accordingly, users need to copy data from this file system to either the /projects or the massdata file systems.

Costing:  The cost per TB for this storage reflects the combination of the disk cache, licensing costs, infiniband fabric, and the servers that provide for access.

Expansion:  Only minimal expansion of the current global Lustre filesystem is possible since a single Lustre filesystem is limited by the size of the disk allocated to the storage of metadata at the time of creation.  As a rule of thumb, 1 TB of metadata storage will allow for the file system to grow to 1 PB in size.

Access: Access to the /g/data service is via Lustre (Infiniband) or NFS mounts.

Disaster Recovery (DR):  Disaster recovery is not provided for the /g/data filesystem.

Development Roadmap: Due to the increasing performance requirements of a number of projects, together with the additional storage requirements for collections that are being brought on line (in a number of cases with funding from the Research Data Storage Infrastructure (RDSI) project), work is currently underway (July 2013) to establish a new global Lustre filesystem, the size of which will be 8 PB.

The first 2.2 PB of this system are expected will be online by early August 2013, with expansion planned for the coming months, and reaching full capacity by the end of November 2013. 

In December 2013, an upgrade to Lustre 2.4 is planned, with some rudimentary HSM capability becoming available.  As the majority of this filesystem will be used for existing projects and datasets, work is underway on the creation of new separate Lustre storage system to accommodate the planned RDSI-funded collections, with the first delivery of equipment in July/August 2013.

In Collaboration With