US-German project will work on US supercomputer Discovery
It is another success for the long-standing collaboration between the University of Delaware (USA) and the Helmholtz-Zentrum Dresden-Rossendorf (HZDR): In a competitive bid to gain access to the planned US supercomputer Discovery, the joint project PIConGPU is among the nine that got green-lighted in mid-June. The team’s proposal features an artificial intelligence-powered approach that aims to identify only those high-fidelity (and costly) simulations that are most relevant for solving a given scientific problem. These are the ones that will actually be calculated. Simulations done by PIConGPU ran on Discovery’s predecessor Frontier, currently the third fastest supercomputer in the world. At HZDR, the institutes Center for Advanced Systems Understanding (CASUS) in Görlitz and Institute of Radiation Physics in Dresden co-lead the PIConGPU project.
“We succeeded in the first CAAR program in 2019, when the world’s first exascale computer, Frontier, was built and optimized,” remembers CASUS Research Team Leader Dr. Michael Bussmann. “This unique opportunity really propelled us forward, as it allowed us to prepare PIConGPU early on for exascale computing, which later on resulted in being included in the Research and Early Access Program for JUPITER, the first European exascale computer at Forschungszentrum Jülich in Germany. That’s why we’re incredibly proud to have made it also into the CAAR program for Discovery.” Bussmann hopes that the team will be able, for the first time, to combine large artificial intelligence (AI) models with high-resolution simulations in order to determine the correct operating parameters for new technologies such as laser particle accelerators and laser fusion facilities.
Additional information:
Dr. Michael Bussmann
Research Team Leader
Center for Advanced Systems Understanding (CASUS) at HZDR
Media contact:
Dr. Martin Laqua
Officer Communications, Press and Public Relations
Center for Advanced Systems Understanding (CASUS) at HZDR
Example from the laser particle accelerator field of application: The picture is a still of a PIConGPU simulation where a high-intensity laser beam (red) hits a cryogenic hydrogen jet (blue-yellow-pink for increasing electron energy density). After jet disruption, an expanding cloud of plasma electrons creates strong electric currents and thus electromagnetic fields that give rise to proton acceleration. These accelerated protons could be used for cancer treatment in the future. Source: R. Pausch/HZDR
Laser-driven fusion, in particular, is currently gaining momentum as a potential energy source. A key challenge is the energy transfer from the lasers to the fusion targets. Nanostructured targets are widely considered to boost absorption and energy coupling, but finding the right materials and geometry is challenging. The demand for experiments is many times greater than the capacity available at research facilities. This is where simulations prove to be extremely useful. But the supply-and-demand situation is no different when it comes to high-performance computers. Discovery’s increased performance will allow PIConGPU to run high-fidelity simulations using AI-driven multi-objective optimization. Thanks to this optimization step, only the most informative runs are selected and executed, ensuring that the limited supercomputer capacity is used as efficiently as possible.
AI calculations move from Frontier to Discovery
Discovery will be delivered in 2028 at the latest and is planned to go into operation in 2029. It will have with significantly greater performance than Frontier and it features a computing cluster optimized for AI calculations. This cluster will be installed as early as this year. In combination with Discovery, it will be perfectly suited for AI-assisted simulations. Until Discovery’s launch, the AI cluster will be combined with Frontier. Prof. Sunita Chandrasekaran rejoices: “Given that we have expanded PIConGPU with several AI-based optimizations over the past two years, we had secretly hoped to be selected for the Discovery CAAR program. When we received the notification, we were nevertheless overjoyed.”
Bussmann adds that important progress was also achieved on European infrastructure: “A major factor was that the HZDR is part of the EuroHPC Center of Excellence PlasmaPEPSC. Access to European exascale computers such as JUPITER via PlasmaPEPSC ensures that we always remain at the forefront of research.”
PIconGPU is based on the hardware-agnostic library Alpaka which allows an efficient adoption of existing software to new hardware. PIConGPU is the abbreviation of Particle-In-Cell on Graphics Processing Unit. The open-source code is used by scientists to study laser-plasma interactions in particle accelerator development for radiation therapy of cancer, in high-energy physics or in photon research. PIConGPU has been in continuous development for more than ten years.
Team PIConGPU – the only international project
The other eight projects selected in the call for proposals were, without exception, put forward by applicants from the United States: from industrial companies such as General Electric, national research laboratories such as Caltech, or federal agencies such as NASA. The submissions were evaluated based on their potential for scientific advancements and the ability to solve grand challenge problems designed to push the limits of Discovery’s computational capabilities. Another key requirement was that each project team had to demonstrate that it could utilize the Discovery hardware so effectively that the application will likely run three to five times faster than on Frontier.
The Discovery supercomputer will be built at the Oak Ridge National Laboratory (ORNL), a research and development center based in the town of Oak Ridge (USA). Discovery continues ORNL’s tradition of providing flagship supercomputers, of which it has deployed a total of seven since 2004. Each of the last four machines – Jaguar, Titan, Summit and Frontier – were recognized as the world’s fastest system at the time. Thanks to the collaboration of HZDR scientists with the University of Delaware team of Chandrasekaran, the PIConGPU simulation software had already been granted running time on Summit and Frontier. In the case of Frontier, ORNL partnered with application developers and vendor partners through a Center for Accelerated Application Readiness (CAAR) program.
________________________________________________________
About the Center for Advanced Systems Understanding
CASUS was founded 2019 in Görlitz/Germany and pursues data-intensive interdisciplinary systems research in such diverse disciplines as earth systems research, systems biology or materials research. The goal of CASUS is to create digital images of complex systems of unprecedented fidelity to reality with innovative methods from mathematics, theoretical systems research, simulations as well as data and computer science to give answers to urgent societal questions. The founding partners of CASUS are the Helmholtz-Zentrum Dresden-Rossendorf (HZDR), the Helmholtz Centre for Environmental Research in Leipzig (UFZ), the Max Planck Institute of Molecular Cell Biology and Genetics in Dresden (MPI-CBG), the Technical University of Dresden (TUD) and the University of Wrocław (UWr). CASUS, managed as an institute of the HZDR, is funded by the German Federal Ministry of Research, Technology and Space (BMFTR) and the Saxon State Ministry for Science, Culture and Tourism (SMWK).
Additional information:
Dr. Michael Bussmann
Research Team Leader
Center for Advanced Systems Understanding (CASUS) at HZDR
Media contact:
Dr. Martin Laqua
Officer Communications, Press and Public Relations Center for Advanced Systems Understanding (CASUS) at HZDR