Argonne’s simulation code Nek5000 sheds light on the turbulent flow fields of internal combustion engines, nuclear reactors, airplane wings, and more. The open-source software, which has evolved for over 30 years, features scalable algorithms that are fast and efficient on platforms ranging from laptops to the world’s fastest computers. It has a user community of over 400 scientists and engineers in academia, laboratories and industry.
Integrated with high performance computing resources, Nek5000 can solve computational fluid dynamics problems with remarkable speed and precision. Still, there are areas for improvement, including opportunities to increase the complexity of the geometry and physics handled by the code.
Dr. Paul Fischer, chief architect of Nek5000 and Professor of Computer Science at UIUC, is collaborating with researchers at Argonne and around the world to address these and other improvements. Such improvements will enable Nek5000 to address challenging science and engineering problems on existing as well as well as future computing systems, including Aurora, America’s first exascale system due to arrive at Argonne in 2021.
“The potential impact of this work is an unmatched, accurate and highly scalable tool to facilitate the efficient design of internal combustion engines in a shorter turnaround time,” said Sibendu Som, section manager of multi-physics computation at Argonne.
Improving performance
On existing machines such as Argonne’s Mira, Nek5000 can realize ideal scaling out to a million processes with relatively small problems. “Its scalability matches the performance expected from our model analysis,” notes Fischer, “which means we are able to run as fast as possible on these platforms.” The challenge for future architectures will be to get high performance on sophisticated nodes with many cores per node. The Argonne team is working closely with collaborators at several labs and universities to ensure that Nek5000 will deliver on these future architectures.
Making more accurate predictions
Researchers are also collaborating to improve the accuracy of Nek5000, so that the code is more predictive. This effort includes adapting the code to better handle the complex, moving geometries that comprise a combustion engine, like fuel interjectors and valves.
“Nek5000 is typically used to simulate complex phenomena within non-moving parts, like nuclear reactors, so this work is new kind of application,” Som said. “We are developing Nek5000 to overcome the computational challenges associated with moving geometries so that we can do engine simulations with all the physics represented.”
With Argonne post-doctoral researcher Saumil Patel, the team has developed a new time-advancement algorithm that yields a four-fold reduction in simulation times for high-speed events such as flow through an intake valve. According to Patel, this new technique is currently being used by researchers at the Aerothermochemistry and Combustion Systems Laboratory at ETH-Zürich who have successfully simulated over 30 cycles of an experimental engine on Theta, the Cray XC40 supercomputer at the Argonne Leadership Computing Facility, an Office of Science User Facility.
Potential impact in engine studies
More scalable, more accurate code will ultimately benefit engine research, such as studies of cycle-to-cycle variability, a poorly understood phenomenon. Cycle-to-cycle variability is variation in fuel combustion that occurs from one engine cycle to the next; it’s among the biggest enemies to engine performance, undermining robust operation.
Combustion researchers have spent decades trying to understand this problematic process, so that they can find ways to control or mitigate it. Unfortunately, the micro-events contributing to cycle variability are difficult to reproduce reliably in the experimental realm so researchers rely heavily on modeling and simulation to study this phenomenon.
“Researchers have used Nek5000 to perform simulations that investigate the causes of cycle-to-cycle variability,” said Argonne mechanical engineer Mushin Ameen. “By improving this tool further, making it faster and more accurate, the Argonne team can help them advance their understanding of the complex relationships contributing to cycle-to-cycle variability.”