NCI’s data centre houses our supercomputing, cloud computing and data storage systems. Taking up most of the floor space of the NCI building, the data centre and associated plant room are at the heart of our operations.
The data centre features various innovative electrical distribution and cooling solutions to maintain peak performance for all of our systems. A false floor raises the computing equipment up and leaves room for pipes of hot and cold water, and electrical cables, to reach every corner of the room. Plugged into the servers, fans and power distribution boards from below, these pipes and cables keep the Gadi supercomputer, all of our filesystems and the associated infrastructure running 24/7. In total, more than 2 megawatts of electrical power gets distributed to our systems, with a mix of air- and water-cooling in use throughout.
Large centralised supercomputers such as Gadi provide significant efficiency gains over small distributed clusters. A large system benefits from efficient cooling technologies that work best at scale, can stay in a lower-power idle state when not in use, and can run more efficient algorithms and scientific code. The latest generation of supercomputer processors are more efficient than any before them, performing the same calculations using less electricity. Data storage electricity use is also minimised by efficient use of magnetic tape for large and archival data storage. Direct liquid cooling with warm water as used in the Gadi supercomputer requires less energy for cooling the machine than previous air-cooled systems, and evaporative cooling is a further extremely efficient way of removing excess heat from the system
Gadi uses innovative water-cooling technology, with water piped directly over the processors and pulling the excess heat away quickly and efficiently. For a supercomputer of Gadi's size, combining water and air cooling helps to keep it cool and operating at optimal temperatures. This technology can reduce data centre power costs by up to 40%. Both the water and the air cooling systems are set up so that heat generated from the computers is concentrated in circulating water pipes, thereby removing the heat from the data centre air and importantly, the processors. The hot water is then piped to the data centre roof from where it evaporates. The warm-water cooling technology removes the need for electrical chillers to assist with remove excess heat from the Gadi supercomputer.
Canberra’s low humidity and relatively low temperatures makes it an ideal place to employ free cooling. NCI continues to improve and optimise the electrical and cooling efficiency of our data centre, striving to have as efficient and effective a cooling system as possible. This has been a key design feature of the building since its inauguration in late 2012. By leveraging the cooling properties of evaporating water, free cooling efficiently removes excess heat from the processors, hard drives and server racks.