UNSW PhD candidate, visiting student at the Garvan Institute of Medical Research and bioinformatics tool developer Hasindu Gamaarachchi is using NCI’s heterogenous architecture – its integrated mix of CPU and GPU processors– to explore efficient large-scale genomics studies that have not been possible before.
Mr Gamaarachchi has been looking at ways to accelerate nanopore sequencing on embedded systems, which has the potential to transform modern-day genomics by enabling portable and real-time sequencing applications. Such a breakthrough could enable point-of-care genetic-based diagnostics for patients and in-the-field genotyping. The other end of nanopore sequencing is for population-scale studies facilitated by an ultra-high throughput sequencer called the PromethION. Large research labs such as the Garvan Institute of Medical Research using these sequencers can sequence up to 48 human genome samples in parallel. Nanopore PromethION sequencers generate an enormous amount of data, up to four terabytes of data per sample.
Currently, analysing raw nanopore signal data from a PromethION is computationally taxing – the existing software can take weeks to process a single sample when run on a high-performance computer. Clearly, the computational resources that would be required for processing 48 samples are infeasible for regular large-scale operations.
The research team is investigating a hybrid solution that uses both CPUs and GPUs to realise fast and efficient data processing, and take advantage of the benefits of each type of processor. GPUs in particular provide a novel, highly parallel architecture for efficient processing of certain kinds of data. For this, the team are using NCI to benchmark the Adaptive Banded Event Alignment (ABEA) algorithm running on the latest Nvidia V100 GPUs available on the Gadi system.
“From a technical perspective, we had already demonstrated that the CPU-GPU approach can perform around 5 times faster than the CPU-only method,” explains Mr Hasindu.
“Soon we realised that our bioinformatics software named f5c, a massively re-engineered version of the existing Nanopolish software that incorporates the CPU-GPU optimised ABEA, is highly efficient for use in large scale analyses that involve processing sequencing data from the PromethION sequencers. At the moment, we are doing some benchmarks on several such large datasets. This is where we are using Gadi resources.”
“We wanted to benchmark f5c on a representative environment and the NCI cluster was a very suitable one. As anticipated, the original parameters for a single HPC node were not ideal for this environment. However, by tuning the performance-governing parameters to suit the environment, we managed to process a single PromethION sample in less than 15 hours. We believe that with the number of nodes NCI has, processing 48 PromethION samples at once is reasonably possible now. We are currently performing benchmarks on a few different samples to further consolidate the efficacy of the identified parameters.”
NCI has 160 nodes that in total feature 640 Nvidia Tesla Volta V100 GPU accelerators. These GPU nodes also include dual 24-core Intel Xeon Cascade Lake Platinum processors, making NCI an ideal environment for benchmarking algorithms requiring a combination of CPU and GPU resources. In this regard, NCI is in a unique position to support Mr Hasindu and his team in this critical development of new genomics software.
