Researchers use JUWELS for record-breaking simulations of turbulence's smallest structures

When you pour cream into a cup of coffee, the viscous liquid seems to lazily disperse throughout the cup. Take a mixing spoon or straw to the cup, though, and the cream and coffee seem to quickly and seamlessly combine into a lighter color and, at least for some, a more enjoyable beverage.

The science behind this relatively simple anecdote actually speaks to a larger truth about complex fluid dynamics and underpins many of the advancements made in transportation, power generation, and other technologies since the industrial era--the seemingly random chaotic motions known as turbulence play a vital role in chemical and industrial processes that rely on effective mixing of different fluids.

While scientists have long studied turbulent fluid flows, their inherent chaotic natures have prevented researchers from developing an exhaustive list of reliable "rules," or universal models for accurately describing and predicting turbulence. This tall challenge has left turbulence as one of the last major unsolved "grand challenges" in physics.

In recent years, high-performance computing (HPC) resources have played an increasingly important role in gaining insight into how turbulence influences fluids under a variety of circumstances. Recently, researchers from the RWTH Aachen University and the CORIA (CNRS UMR 6614) research facility in France have been using HPC resources at the Jülich Supercomputing Centre (JSC), one of the three HPC centres comprising the Gauss Centre for Supercomputing (GCS), to run high-resolution direct numerical simulations (DNS) of turbulent setups including jet flames. While extremely computationally expensive, DNS of turbulence allows researchers to develop better models to run on more modest computing resources that can help academic or industrial researchers using turbulence's effects on a given fluid flow.

"The goal of our research is to ultimately improve these models, specifically in the context of combustion and mixing applications," said Dr. Michael Gauding, CORIA scientist and researcher on the project. The team's recent work was just named the distinguished paper from the "Turbulent Flames" colloquium, which happened as part of the 38th International Symposium on Combustion.

Starts and stops

Despite its seemingly random, chaotic characteristics, researchers have identified some important properties that are universal, or at least very common, for turbulence under specific conditions. Researchers studying how fuel and air mix in a combustion reaction, for instance, rely on turbulence to ensure a high mixing efficiency. Much of that important turbulent motion may stem from what happens in a thin area near the edge of the flame, where its chaotic motions collide with the smoother-flowing fluids around it. This area, the turbulent-non-turbulent interface (TNTI), has big implications for understanding turbulent mixing.

While running their DNS calculations, Gauding and his collaborator, Mathis Bode from RWTH Aachen, set out to specifically focus on this some of the subtler, more complex phenomena that take place at the TNTI.

Specifically, the researchers wanted to better understand the rare but powerful fluctuations called "intermittency" - an irregular process happening locally but with very high amplitude. In turbulent flames, intermittency enhances the mixing and combustion efficiency but too much can also extinguish the flame. Scientists distinguish between internal intermittency, which occurs at the smallest scales and is a characteristic feature of any fully developed turbulent flow, and external intermittency, which manifests itself at the edge of the flame and depends on the structure of the TNTI.

Even using world-class HPC resources, running large DNS simulations of turbulence is computationally expensive, as researchers cannot use assumptions about the fluid motion, but rather solve the governing equations for all relevant scales in a given system--and the scale range increases with the "strength" of turbulence as power law. Even among researchers with access to HPC resources, simulations oftentimes lack the necessary resolution to fully resolve intermittency, which occurs in thin confined layers.

For Bode and Gauding, understanding the small-scale turbulence happening at the thin boundary of the flame is the point. "Our simulations are highly resolved and are interested in these thin layers," Bode said. "For production runs, the simulation resolution is significantly higher compared to similar DNS simulations to accurately resolve the strong bursts that are connected to intermittency."

The researchers were able to use the supercomputers JUQUEEN, JURECA, and JUWELS at JSC to build a comprehensive database of turbulence simulations. For example, one simulation was run for multiple days on the full JUQUEEN module, employing all 458,752 compute cores during the centre's "Big Week" in 2019, simulating a jet flow with about 230 billion grid points.

Mixing and matching

With a better understanding of the role that intermittency plays, the team takes data from their DNS runs and using it to improve less computationally demanding large eddy simulations (LES). While still perfectly accurate for a variety of research aims, LES are somewhere between an ab initio simulation that begins with no assumptions and a model that has already baked in certain rules about how fluids will behave.

Studying turbulent jet flames has implications for a variety of engineering goals, from aerospace technologies to power plants. While many researchers studying fluid dynamics have access to HPC resources such as those at JSC, others do not. LES models can often run on more modest computing resources, and the team can use their DNS data to help better inform these LES models, making less computationally demanding simulations more accurate. "In general, present LES models are not able to accurately account for these phenomena in the vicinity of the TNTI," Gauding said.

The team was able to scale its application to take full advantage of JSC computing resources partially by regularly participating in training events and workshops held at JSC. Despite already being able to leverage large amounts of HPC power, though, the team recognizes that this scientific challenge is complex enough that even next-generation HPC systems capable of reaching exascale performance--slightly more than twice as fast as today's fastest supercomputer, the Fugaku supercomputer at RIKEN in Japan--may not be able to fully simulate these turbulent dynamics. However, each computational advancement allows the team to increase the degrees of freedom and include additional physics in their simulations. The researchers are also looking at using more data-driven approaches for including intermittency in simulations, as well as improving, developing, and validating models based on the team's DNS data.