Authors

Abstract

This paper presents our work toward correct and efficient automatic differentiation of OpenMP parallel worksharing loops in forwardand reverse mode. Automatic differentiation is a method to obtain gradients of numerical programs, which are crucial in optimization,uncertainty quantification, and machine learning. The computational cost to compute gradients is a common bottleneck in practice. Forapplications that are parallelized for multicore CPUs or GPUs using OpenMP, one also wishes to compute the gradients in parallel. Wepropose a framework to reason about the correctness of the generated derivative code, from which we justify our OpenMP extensionto the differentiation model. We implement this model in the automatic differentiation tool Tapenade and present test cases that aredifferentiated following our extended differentiation procedure. Performance of the generated derivative programs in forward andreverse mode is better than sequential, although our reverse mode often scales worse than the input programs