DIY is an open-source package of scalable building blocks for data movement tailored to the needs of large-scale parallel analysis workloads.

Scalable, parallel analysis of data-intensive computational science relies on the decomposition of the analysis problem among a large number of distributed-memory compute nodes, the efficient data exchange among them, and data transport between compute nodes and a parallel storage system.

Configurable data partitioning, scalable data exchange, and efficient parallel I/O are the main components of DIY, a library that assists developers in parallelizing serial analysis algorithms by providing configurable, high-performance data movement algorithms built on top of MPI. Computational scientists, data analysis researchers, and visualization tool builders can all benefit from these tools.