Using sophisticated data reduction techniques is important in order to reduce the size of the scientific data while keeping enough information for scientific discovery. Unfortunately, lossless compres- sion is rarely effective on scientific datasets. Only lossy compression with user-set error controls can help significantly reduce scientific dataset sizes while respecting user-requested accuracy.

To address this challenge, we are developing EZ, a lossy compressor for scientific datasets respecting user-set error controls. Specifically, EZ will improve the SZ core compression algorithms using more sophisticated segmentation techniques to guarantee maximal compression factor in all cases, more sophisticated linearization techniques to improve data locality, more data transformations to improve data predictability, and more lossless compression algorithms to further improve the compression factors.

EZ will be designed to cover a large spectrum of DOE applications by offering excellent compression performance and a variety of error controls. Planned collaborations include working with developers of the HAAC (cosmology) and ACME (climate) simulation codes.