Argonne National Laboratory
Argonne National Laboratory has a comprehensive program in high energy physics that encompasses the Energy Frontier (ATLAS at LHC), Intensity Frontier (neutrino and muon experiments — NOvA, Mu2e, g-2), and Cosmic Frontier (computational cosmology and surveys — DES, DESI, LSST, SPHEREx, SPT, WFIRST). Advanced computation plays a significant role in the frontier science that Argonne conducts. The ATLAS group in Argonne’s HEP division hosts the ATLAS Analysis Support Center, while the Computing group focuses on the use of high-performance computing for LHC data analysis, detector simulation, and perturbative QCD.
The Computational Cosmology group performs some of the largest cosmological simulations in the world. Argonne is home to the Argonne Leadership Computing Facility (ALCF), one of two such facilities in the DOE complex. ALCF staff carry out front-line research in HEP-relevant areas such as computational cosmology and lattice QCD. The ALCF hosts the 10-Petaflop IBM BG/Q system Mira with 750K+ cores; another 100K BG/Q cores reside in the smaller systems Cetus and Vesta. The CPU/GPU analysis cluster Cooley offers 300 Teraflops of performance. Theta, an 8- Petaflop Intel Xeon Phi Knights Landing-based system, arrived in 2016. HEP Division researchers work closely with ALCF staff, as well as with members of Argonne’s Mathematics and Computer Science (MCS) division on cosmological simulations.
Lawrence Berkeley National Laboratory
In the 21st century, particle physics and cosmology have come together in a unified quest to understand the fundamental constituents of the Universe and how they evolved after the Big Bang. The Lawrence Berkeley National Laboratory (LBNL), the Physics Division carries out research that spans the full range of particle physics and cosmology, from studies of sub-atomic particles created in accelerator collisions that decay almost instantly, to large-scale cosmological structures observed in galaxy surveys looking back billions of years in cosmic time. We are involved in several HEP experiments and play lead roles in the computational analysis and simulation work for: ATLAS, DES, DESI, LSST-DESC, LZ, LBNE, Daya Bay and Mu2e. LBNL is also host to the National Energy Research Scientific Computing Center (NERSC), the primary scientific computing facility for the Office of Science in the U.S. Department of Energy. As one of the largest facilities in the world devoted to providing computational resources and expertise for basic scientific research, NERSC is a world leader in accelerating scientific discovery through computation. NERSC is host to two large HPC systems Edison, a 130+k-core Cray XC30; and Cori, a 500+k-core Cray system based on the second generation of Intel Xeon Phi’s called Knights Landing.
Fermilab National Accelerator Laboratory
Fermilab is the sole DOE national laboratory devoted exclusively to particle physics and, as such, has a special place in the HEP landscape, where it is making significant contributions in each of the scientific frontiers. In terms of the official DOE Office of Science list of Core Competencies, Fermilab is recognized in the area of Advanced Computer Science, Visualization, and Data. Fermilab has a significant computing facility housing 80,000 cores of compute along with 30 petabytes of cache and nearly an exabyte of robotic tape storage capacity. Two 100-gigabit links connect this facility to the ESnet science network in a robust fashion. Fermilab hosts the CMS Tier 1 computing center along with a large analysis cluster to support the LHC Physics Center (LPC) located on site. Fermilab is home to the Intensity Frontier and is supporting MINOS, Minerva, Microboone, LArIAT, LAr1ND, ICARUS, g-2, Mu2e, NOvA and DUNE. Fermilab is also playing a key role in both the dark energy and dark matter experiments, where it is engaged in DES, DESI, LSST for dark energy and CDMS, COUPP, and Darkside for dark matter efforts. Supporting so many experiments, the scientific computing division (140 strong) has developed a series of common tools, workflows, data acquisition systems, etc. to enable the experiments to focus on the science rather than on the computational infrastructure.
Brookhaven National Laboratory
Brookhaven National Laboratory (BNL) is a multipurpose research institution located on Long Island, New York, bringing world-class facilities and expertise to the most exciting and important questions in basic and applied science — from the birth of our universe to the sustainable energy technology of tomorrow. We operate cutting-edge, large-scale facilities (NSLS II, CFN, ATF, ARM, RHIC, US Atlas Tier 1, Belle II Computing, US QCD Computing) for studies in physics, chemistry, biology, medicine, applied science, and a wide range of advanced technologies. Our award-winning history, including seven Nobel Prizes, stretches back to 1947, and we continue to unravel mysteries from the nanoscale to the cosmic scale, and everything in between. Building on its capabilities in computational science and data management, the BNL Computational Science Initiative (CSI) was established in 2015 to integrate and leverage computer science, computational science, and applied mathematics expertise and investments across multiple programs at the Laboratory. In providing leading computational and data science solutions for BNL’s flagship facilities CSI is confirming Brookhaven as a leader in tackling the “big data” challenges at the frontiers of scientific discovery. The associated Scientific Data and Computing Center (SDCC) houses with over 100 PB the 2nd largest scientific data collection in the US and the 4th largest worldwide. SDCC processes between 400-500 PB of largely physics-related data annually, supported by a wide variety of high-throughput, high-performance, and cloud computing resources. CSI holds regularly training courses, workshops, and hackathon’s to enable its attendees to effectively leverage the newest computing architectures and programming paradigms for their research.