The National Computational Infrastructure in Canberra is set to become the master repository of the South-East Asia/South Pacific regional data hub of Copernicus, Europe's ambitious and multi-faceted Earth observation programme.
Envisat image of the Great Barrier Reef off Australia's Queensland coast. (ESA)
In 2015, an agreement between the Australian Government (through Geoscience Australia) and the European Commission cemented Australia's critical role in the acquisition and processing of Copernicus data pertaining to the South East Asia-South-Pacific region. Over 220 terabytes of regional data is already available, a number which is expected to increase to over 7 petabytes in the next ten years.
The Copernicus program collects vast amounts of global data from Europe's Sentinel satellites, a growing fleet orbiting the Earth and capturing new measurements every day. This data is then stored, analysed and distributed for a wide range of applications such as environmental protection, sustainable resource development, climate change mitigation and managing risks and emergency response for natural disasters.
The regional Copernicus data hub, for which NCI will act as the master repository, will greatly improve access to Earth observation data in the South East Asia/South Pacific region, including but not limited to sea temperature, atmospheric composition and vegetation health. These efforts to enhance access to this data come at a critical time; as the region undergoes massive changes in population and urban density, those living within it are expected to face several great challenges, including the protection of environmental assets, promotion of sustainable natural resource development, and risk reduction for natural disasters.
NCI is home to the fastest filesystems in the Southern Hemisphere, the highest performing research cloud, and Australia's first petaflop supercomputer. This unique colocation of data, cloud and HPC resources at NCI will permit the easy manipulation of Copernicus data via these interconnected resources. For example, NCI's supercomputer will allow researchers to create new and improved climate models using Copernicus data already on-site, negating the need for lengthy data transfers.
The addition of data from the Copernicus mission follows NCI's long history of hosting and managing Earth observation data, including over 700 terabytes from the United States Government's decades-long LANDSAT programme, and approximately 353 terabytes from Japan's Himawari weather satellites.
The Copernicus data set will soon eclipse all previous Earth observation data sets, becoming a primary resource for research organisations and other institutions in the region.
The expert team at NCI are using their wide-area network expertise to manage the complex paths that are used to dispatch Copernicus data from multiple long-distance sources. For example, data originating from the European Space Agency will cross three distinct network carriers and take over 330 milliseconds for one data packet to traverse the paths. This data originates at ESA's IntHub, a dedicated system for strategic international partners in Copernicus, and passes through the GÈANT, Internet2, PacificWave and AARNET networks using a 'pull' model.
Meanwhile, EUMETSAT data is 'pushed' to NCI using EUMETCast, a TCP multicast system. Two separate intercontinental data paths ensure redundancy for data transmission.
The availability of the data will occur through several worldwide research and education network infrastructures, initially through the pan-European GÈANT network, and in Australia via AARNet, in partnership with Geoscience Australia.
This new NCI resource has been made possible with the assistance of a consortium that includes CSIRO, Geoscience Australia and the state governments of Western Australia, New South Wales and Queensland, as well through a partnership with the European Commission, European Space Agency and EUMETSAT.
