• Home
• People
• Jobs
• Visit
• Thesis Topics
• Publications
• Conference Rankings
• Research
• Bcast
• Teaching
• Contact

SPONSORS

COLLABORATIONS

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

A. Boudaoud, A. Calotoiu, M. Copik, T. Hoefler:
	DaCe AD: Unifying High-Performance Automatic Differentiation for Machine Learning and Scientific Computing (In Proceedings of the IEEE International Conference on Cluster Computing (CLUSTER), presented in Edinburgh, United Kingdom, pages 1-13, IEEE Press, ISBN: 979-8-3315-3019-8, Sep. 2025) Publisher Reference Abstract Automatic differentiation (AD) is a set of techniques that systematically applies the chain rule to compute the gradients of functions without requiring human intervention. Although the fundamentals of this technology were established decades ago, it is experiencing a renaissance as it plays a key role in efficiently computing gradients for backpropagation in machine learning algorithms. AD is also crucial for many applications in scientific computing domains, particularly emerging techniques that integrate machine learning models within scientific simulations and schemes. Existing AD frameworks have four main limitations: limited support of programming languages, requiring code modifications for AD compatibility, limited performance on scientific computing codes, and a naive store-all solution for forward-pass data required for gradient calculations. These limitations force domain scientists to manually compute the gradients for large problems. This work presents DaCe AD, a general, efficient automatic differentiation engine that requires no code modifications. DaCe AD uses a novel ILP-based algorithm to optimize the trade-off between storing and recomputing to achieve maximum performance within a given memory constraint. We showcase the generality of our method by applying it to NPBench, a suite of HPC benchmarks with diverse scientific computing patterns, where we outperform JAX, a Python framework with state-of-the-art general AD capabilities, by more than 92 times on average without requiring any code changes. Documents download article: access preprint on arxiv: download slides:
	BibTeX
@inproceedings{boudaoud2025dace,   author={Afif Boudaoud and Alexandru Calotoiu and Marcin Copik and Torsten Hoefler},   title={{DaCe AD: Unifying High-Performance Automatic Differentiation for Machine Learning and Scientific Computing}},   year={2025},   month={09},   pages={1-13},   booktitle={Proceedings of the IEEE International Conference on Cluster Computing (CLUSTER)},   location={Edinburgh, United Kingdom},   publisher={IEEE Press},   isbn={979-8-3315-3019-8},   doi={10.1109/CLUSTER59342.2025.11186465}, }