Authors
Foster, Ian; Ainsworth, Mark; Bessac, Julie; Cappello, Franck; Choi, Jong; Di, Sheng; Di, Zichao; Gok, Ali Murat; Guo, Hanqi; Huck, Kevin; Peterka, Tom; Shu, Tong; Wozniak, Justin; Yakushin, Igor; Munson, Todd
Abstract
A growing disparity between supercomputer computation speeds and I/O rates means that it is rapidly becoming infeasible toanalyze supercomputer application output only after that output has been written to a file system. Instead, data-generatingapplications must run concurrently with data reduction and/or analysis operations, with which they exchange information viahigh-speed methods such as interprocess communications. The resulting parallel computing motif, online data analysis andreduction (ODAR), has important implications for both application and HPC systems design. Here we introduce the ODARmotif and its co-design concerns, describe a co-design process for identifying and addressing those concerns, present tools thatassist in the co-design process, and present case studies to illustrate the use of the process and tools in practical settings.