The Second Phase of NCI
The second phase of NCI’s development commenced relatively early, with announcements in the Commonwealth Government Budget of May 2009 of the new Super Science initiatives. Each of the Super Science investments, including the Climate HPC Centre Project through ANU/NCI, was funded from the Education Investment Fund, one of Australia’s Nation Building Funds.
The project objectives were:
- The establishment of a petascale HPC facility prioritised to the needs of research in climate change, earth system science, national water management, which simultaneously would support meritorious and high-impact research in all fields that required access to capability-class computing;
- The identification and support of data-intensive and flagship science application aligned with other government research and infrastructure investments;
- The development and implementation of an access model that would meet the priority use requirements, as well as providing open access on research merit to researchers at publicly-funded research organisations;
- The construction of a purpose-built data centre to house the HPC facility, capable which could be upgraded to handle future usage for at least five years;
The Collaboration Agreement establishes the objectives for the collaboration, puts in place its governance, lays out the access model, and underpins NCI’s business model and planning. This collaboration, which includes four national organisations, together leveraging of the Commonwealth’s infrastructure monies through substantial co-investment, are two distinctive features of NCI.
The Collaboration Agreement set foundations for the implementation of this current phase of NCI and formed the backdrop for an organisational change, from the beginning of 2012, by which the University brought together the NCI Project Office and the ANU Supercomputing Facility into a single operating unit. Under this governance model, ANU governs the operations of NCI on the advice of the NCI Board, to within the limits of its Statutes and policies.
With the Collaboration Agreement in place from July 2011, the data centre construction was initiated in August 2011, and was completed in September 2012, with the building formally handed over to ANU in November of that year. Concurrently, the tender for the HPC system also proceeded through a public tender from August 2011. The contract for the procurement was executed in mid-June 2012, leading to the debut of the system, now known as Raijin, on the Top500 list in November 2012 at rank 24, with a peak performance (Rmax) of 940 Teraflops (for the 93 per cent of the system that had been implemented at the time). Early user service began in late April 2013, with full user service from mid-June 2013. The Fujitsu contract also implemented a most significant collaboration framework to take NCI and its partners into the future, and, in particular, to deliver optimal value for the peak system procurement. This was done through a program of work to optimise the implementation and performance of Australia’s climate and earth system science modelling suite, ACCESS, on Raijin.
NCI’s Board took the strategic decision in 2011 that the NCI Collaboration, which had been established to support operations of a petascale supercomputer, should broaden its scope to provide the comprehensive and integrated services that had long been envisaged. The build-out of the integrated infrastructure solution was undertaken during 2013, with storage procurements from SGI and DDN, and with the establishment of a 3,200 core high-performance cloud from Dell.
The story since 2013 is told incrementally in NCI’s Annual Reports. These provide updates on Australian research outcomes and impacts that have been enabled by the NCI platforms and its expert support team, and the evolution of the infrastructure platform—with further incrementation of the storage from Netapp in 2015, a 40% upgrading of NCI’s supercomputer capacity in 2016 with a 22,000 core Lenovo NeXtScale (Broadwell) system co-funded by a substantial allocation from the NCRIS Agility Fund and the NCI Collaboration, together with major storage upgrades funded from the same source.
The next major stage in the NCI Years is marked by the procurement and installation of Raijin's successor supercomputer, a significantly increased machine named Gadi and comprising latest generation technologies from Intel, Nvidia, Mellanox and others. First appearing on the November 2019 Top500 list in its initial, half-built configuration at number 47, its first fully operational Top500 benchmark was released in June 2020 and placed it at number 24 with a peak performance of 9.26 Petaflops. Comprising more than 150,000 compute cores and 640 GPU accelerators, Gadi underpins the next generation of data and compute intensive research in Australia. Gadi is so-named because it means "to search for" in the language of the Ngunnawal people, the traditional owners of the land that NCI is built on.
Since its beginning, NCI has evolved as the national, high-end research computing service—one which is in the vanguard of international advanced computing, delivering solutions that encompass computational modelling and the needs of big data, that enable research of excellence and impact, which deliver in the national benefit, and which maintain Australian’s international competitiveness in research and innovation.
The collaboration that underpins NCI has grown in strength since its formation, anchored by the Australian National University as NCI’s host, and three of Australia’s national science agencies—the Australian Bureau of Meteorology, the national science agency, CSIRO, and the national geoscience agency, Geoscience Australia. Today, this partnership includes many of Australia’s research universities, or consortia thereof, medical research institutes, and the Australian Research Council, and sustains a large portion of the annual recurrent costs, with contributions from the Australian Government’s National Collaborative Research Infrastructure Strategy (NCRIS) providing the remainder.
Through its tightly-coupled, high-performance computing and data platforms, overlaid with internationally renowned expertise (~60 staff) in computational science, data science and data management, NCI provides essential services that underpin the requirements of research and industry, today and into the future.
Today, NCI plays a pivotal role in Australia’s research and innovation system, supporting the work of more than 5,000 researchers across more than 500 projects being undertaken in 35 universities, 5 national science agencies (including CSIRO, Bureau of Meteorology, and Geoscience Australia), 3 medical research institutes, and industry.
R&D which is reliant on NCI spans the full spectrum—from fundamental research, through the strategic and applied, and on to actual industrial applications. Today, NCI services are both necessary and influential in enhancing the competitiveness and impact of outcomes in every field of science and technology. An increasing number of fields are now highly dependent on the fusion of “big compute” and “big data” that NCI provides—in weather and climate science, the earth sciences, earth observation and remote sensing, medical research, and astronomy, amongst others.
Within the university sector, NCI provides the essential high-performance computing and data foundation for more than 200 research projects, ARC and NHMRC Centres of Excellence and Industry Hubs, and fellowships. Funding for these projects from the Australian Research Council and the National Health and Medical Research Council totals around $60 million per annum, or approximately $250 million over the lifetimes of these projects.
In the domain of the national science agencies, notably BoM, CSIRO and GA, NCI provides critical program-level support in earth system science, by serving as the development platform of the Australian national weather and climate modelling suite, ACCESS. NCI is also a national hub for major national and international satellite earth observation collections, used in the earth, marine and environmental sciences, agriculture for research, the informing of policy development, and the development of important information products for primary industry.
