• Home
• People
• Jobs
• Visit
• Thesis Topics
• Publications
• Conference Rankings
• Research
• Bcast
• Teaching
• Contact

SPONSORS

COLLABORATIONS

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

L. Huang, L. Fusco, F. Scheidl, J. Zibell, M. Armand Sprenger, S. Schemm, T. Hoefler:
	Error bounded compression for weather and climate applications (arXiv:2510.22265. Oct. 2025) Abstract As the resolution of weather and climate simulations increases, the amount of data produced is growing rapidly from hundreds of terabytes to tens of petabytes. The huge size becomes a limiting factor for broader adoption, and its fast growth rate will soon exhaust all the available storage devices. To address these issues, we present EBCC (Error Bounded Climate-data Compressor). It follows a two-layer approach: a base compression layer using JPEG2000 to capture the bulk of the data with a high compression ratio, and a residual compression layer using wavelet transform and SPIHT encoding to efficiently eliminate long-tail extreme errors introduced by the base compression layer. It incorporates a feedback rate-control mechanism for both layers that adjusts compression ratios to achieve the specified maximum error target. We evaluate EBCC alongside other established compression methods on benchmarks related to weather and climate science including error statistics, a case study on primitive and derived variables near a hurricane, evaluation of the closure of the global energy budget, and a Lagrangian air parcel trajectory simulation. This is the first time that trajectory simulation is used to benchmark compression methods. Our method concentrates most errors near zero, while others tend to distribute errors uniformly within the error bound. EBCC outperforms other methods in the benchmarks at relative error targets ranging from 0.1% to 10% and achieves compression ratios from 15x to more than 300x. In the energy budget closure and Lagrangian trajectory benchmarks, it can achieve more than 100x compression while keeping errors within natural variability derived from ERA5 uncertainty members. This verifies the effectiveness of EBCC in creating heavily compressed weather and climate datasets suitable for downstream applications. The source code of EBCC is available in github.com/spcl/EBCC . Documents download article: access preprint on arxiv:
	BibTeX
@article{huang2025error,   author={Langwen Huang and Luigi Fusco and Florian Scheidl and Jan Zibell and Michael Armand Sprenger and Sebastian Schemm and Torsten Hoefler},   title={{Error bounded compression for weather and climate applications}},   journal={arXiv:2510.22265},   year={2025},   month={10},   doi={10.48550/arXiv.2510.22265}, }