openPMD [1] is an open meta-data schema that provides meaning and self-description for datasets in science and engineering. The openPMD API [2], developed openly in collaboration of the Center for Advanced Systems Understanding, Helmholtz-Zentrum Dresden-Rossendorf and Lawrence Berkeley National Laboratory, is an IO middleware library that assists domain-scientists with data description along the openPMD standard for FAIR particle-mesh data, used already in numerous physics simulations [3].
In comparison to direct implementations of application codes against high-speed IO backends, using the openPMD API saves thousands of lines of code per application, reduces integration time for application developers, promotes sharing of best practices for IO library tuning options, provides a high-level interface to describe scientific data and conserves this standardized meta-data for analysis, visualization and coupling workflows [3].
The openPMD-api implements high-speed backends such as HDF5 and ADIOS2, thus enabling Exascale-level IO efficiency for scientific codes. The IO backends can be picked and configured at runtime in order to serve IO adaptability up to the choice of streaming IO (via ADIOS2). Serial JSON and TOML backends are available for prototyping and learning purposes, as well as for use cases with ASCII in-/output.
Additional language bindings on top of the core C++ implementation ease integration into many established post-processing frameworks, such as Python bindings for parallel readers into ParaView, Dask and domain-specific analysis tools.
Further information:
[1] https://github.com/openPMD/openPMD-api/
[2] https://openpmd.org
[3] https://github.com/openPMD/openPMD-projects
References:
Axel Huebl et al. openPMD: A meta data standard for particle and mesh based data. https://doi.org/10.5281/zenodo.591699. 2015. doi: 10.5281/zenodo.591699. url: https://github.com/openPMD.
Fabian Koller et al. openPMD-api: C++ & Python API for Scientific I/O with openPMD. 2018. doi: 10.14278/rodare.27. url: https://github.com/openPMD/openPMD-api.
Franz Poeschel et al. Transitioning from File-Based HPC Workflows to Streaming Data Pipelines with openPMD and ADIOS2. https://doi.org/10.1007%2F978-3-030-96498-6_6. 2022. doi: 10.1007/978-3-030-96498-6_6.