Abstract: Discovering solid and effective solutions to address important societal problems like pandemic, climate change, and transition to green energy often relies on using ultra precise scientific instruments and large-scale numerical simulations that generate extreme volumes of data at high velocity. The next generation of scientific equipment, currently under construction, will generate more scientific data than can be stored, communicated, and analyzed. To respond to this unprecedented challenge, the community has identified scientific data reduction as a major research topic with the goal of finding solutions to reduce data size from one to several orders of magnitude while preserving the potential for scientific discoveries. Several teams are actively searching scientific data reduction techniques satisfying researchers constraints in terms of accuracy, speed, and reduction ratio.

This talk will focus on the SZ lossy compression framework that has just received an R&D100 award. What makes SZ unique is its flexibility and the breadth of its application use-cases. SZ is more than a lossy compressor for scientific data. It is a scientific data reduction platform with modules that can be composed, adapted, and tuned to fit specific scientific data reduction constraints. We will present several use-cases and applications of SZ, including in crystallography and ptychography. We will also discuss techniques to assess the impact of lossy compression on scientific data and analysis.