Student Cluster Competition        

Benchmarks

LINPACK Benchmark

The Linpack Benchmark is a measure of a computer’s floating-point rate of execution. It is determined by running a computer program that solves a dense system of linear equations. It is used by the TOP 500 as a tool to rank peak performance. The benchmark allows the user to scale the size of the problem and to optimize the software in order to achieve the best performance for a given machine. This performance does not reflect the overall performance of a given system, as no single number ever can. It does, however, reflect the performance of a dedicated system for solving a dense system of linear equations. Since the problem is very regular, the performance achieved is quite high, and the performance numbers give a good correction of peak performance.

HPCG Benchmark

The High Performance Conjugate Gradients (HPCG) Benchmark project is an effort to create a new metric for ranking HPC systems. HPCG is intended as a complement to the High Performance LINPACK (HPL) benchmark, currently used to rank the TOP500 computing systems. The computational and data access patterns of HPL are still representative of some important scalable applications, but not all. HPCG is designed to exercise computational and data access patterns that more closely match a different and broad set of important applications, and to give incentive to computer system designers to invest in capabilities that will have impact on the collective performance of these applications.

IO500 Benchmark

The IO500 benchmark is a benchmark suite for High-Performance IO. It harnesses existing and trusted open-source benchmarks such as IOR and MDTest and bundles execution rules and multiple workloads with the purpose to evaluate and analyze the storage devices for various IO patterns. The IO500 benchmark is designed to provide performance boundaries of the storage for HPC applications regarding data and metadata operations under what are commonly observed to be both easy and difficult IO patterns from multiple concurrent clients. Moreover, there is a phase that scans for previously-created files that match certain conditions using a (possibly file system-specific) parallel find utility to evaluate the speed of namespace traversal and file attribute retrieval. The final score that is used to rank submissions in the list is a combined score across all the executed benchmarks.

Applications

Cardioid

Cardioid is a cardiac simulation suite for simulating clinical cardiac phenomena. It is capable of simulating both electrophysiological and mechanical organ-level simulations, and has processing tools for computing cardiac meshes, reconstructions of torso ECGs, and generating realistic cardiac fiber orientations. The Cardioid electrophysiology solver was a Gordon Bell finalist and has strong-scaled to all of the Vulcan supercomputer for a clinically relevant problem. The code is parallelized using MPI, and has separate optimized loops for taking advantage of OpenMP, SIMD instruction sets for CPU architectures, and CUDA for GPU architectures.

Quantum ESPRESSO

Quantum ESPRESSO is a software package for first-principles electronic-structure calculations and materials modeling based on density-functional theory, plane wave basis sets, and pseudopotentials

Mystery Application

At the start of the competition, teams were given an application and datasets for the mystery application, described below. Students are expected to build, optimize and run this mystery application, all at the competition.

CosmicTagger is a multi-image, semantic segmentation application from high energy neutrino physics. With large, high resolution and sparse target labels, it is a challenging segmentation task. When trained to completion, this network achieves more than 95% accuracy on pixels of interest, outperforming traditional techniques by a factor of 5. The high resolution nature of the images, however, makes this application computationally expensive and difficult to train to convergence without a high performance cluster. More information here.

Reproducibility Challenge

Once again, students in the cluster competition will be asked to replicate the results of a publication from the previous year’s SC conference. For this challenge, students will take on the role of reviewing an SC20 paper to see if its results are reproducible. The SC21 Reproducibility Committee has selected the paper “A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery” by Ankit Srivastava, Sriram P. Chockalingam, and Srinivas Aluru to be the Student Cluster Competition (SCC) benchmark for the Reproducibility Challenge this year.

Last year, thanks to the adoption of automatically generating an Artifact Descriptor (AD) during submission time, all accepted papers from SC20 featured an AD in their appendix. A team of reviewers selected the paper from these past papers based on the AD, author interviews, and its suitability for the SCC. The authors and the Reproducibility Committee have been working to create a reproducible benchmark that builds on the paper’s results. During the SCC, the student teams will be asked to run the benchmark, attempting to reproduce the findings from the original paper under different settings with different data sets.