Exploring the Fast Simulation of Large-Scale Neural Networks on GPU Clusters

Published on July 4, 2022

When it comes to understanding the complex dynamics of neuronal populations and their relation to brain function, spiking neural network models are like a giant microscope. These models have been made even more powerful with the use of parallel computing technologies and high-performance clusters. In this study, researchers evaluated the performance of NEST GPU, a CUDA-C/C++ library, on a multi-area model of the macaque cortex. The simulation involved 4 million neurons, 24 billion synapses, and represented a significant surface area of the cortex. The results were compared to those obtained from a CPU-based simulator called NEST. The findings showed that NEST GPU achieved an optimal match with NEST in terms of neural activity measures and outperformed it in terms of simulation time. It was able to simulate a second of biological time 3.1 times faster than NEST on a GPU cluster equipped with NVIDIA V100 GPUs. This study opens up new possibilities for faster and more detailed simulations of large-scale neural networks.

Spiking neural network models are increasingly establishing themselves as an effective tool for simulating the dynamics of neuronal populations and for understanding the relationship between these dynamics and brain function. Furthermore, the continuous development of parallel computing technologies and the growing availability of computational resources are leading to an era of large-scale simulations capable of describing regions of the brain of ever larger dimensions at increasing detail. Recently, the possibility to use MPI-based parallel codes on GPU-equipped clusters to run such complex simulations has emerged, opening up novel paths to further speed-ups. NEST GPU is a GPU library written in CUDA-C/C++ for large-scale simulations of spiking neural networks, which was recently extended with a novel algorithm for remote spike communication through MPI on a GPU cluster. In this work we evaluate its performance on the simulation of a multi-area model of macaque vision-related cortex, made up of about 4 million neurons and 24 billion synapses and representing 32 mm2 surface area of the macaque cortex. The outcome of the simulations is compared against that obtained using the well-known CPU-based spiking neural network simulator NEST on a high-performance computing cluster. The results show not only an optimal match with the NEST statistical measures of the neural activity in terms of three informative distributions, but also remarkable achievements in terms of simulation time per second of biological activity. Indeed, NEST GPU was able to simulate a second of biological time of the full-scale macaque cortex model in its metastable state 3.1× faster than NEST using 32 compute nodes equipped with an NVIDIA V100 GPU each. Using the same configuration, the ground state of the full-scale macaque cortex model was simulated 2.4× faster than NEST.

Read Full Article (External Site)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>