Student Cluster Competition
-
LINPACK Benchmark
http://top500.org/project/linpack/
The Linpack Benchmark is a measure of a computer’s floating-point rate of execution. It is determined by running a computer program that solves a dense system of linear equations. It is used by the TOP 500 as a tool to rank peak performance. The benchmark allows the user to scale the size of the problem and to optimize the software in order to achieve the best performance for a given machine. This performance does not reflect the overall performance of a given system, as no single number ever can. It does, however, reflect the performance of a dedicated system for solving a dense system of linear equations. Since the problem is very regular, the performance achieved is quite high, and the performance numbers give a good correction of peak performance.
- HPCG Benchmark
http://hpcg-benchmark.org/
The High Performance Conjugate Gradients (HPCG) Benchmark project is an effort to create a new metric for ranking HPC systems. HPCG is intended as a complement to the High Performance LINPACK (HPL) benchmark, currently used to rank the TOP500 computing systems. The computational and data access patterns of HPL are still representative of some important scalable applications, but not all. HPCG is designed to exercise computational and data access patterns that more closely match a different and broad set of important applications, and to give incentive to computer system designers to invest in capabilities that will have impact on the collective performance of these applications.
- Reproducibility Challenge
For a second year in a row, students in the cluster competition will be asked to replicate the results of a publication from the previous year's Supercomputing conference.
Many of the sciences have a long tradition of requiring reproduction of results before accepting them in the scientific community. Computer science has lagged other sciences in this respect. The goal of this challenge in the Student Cluster Competition is to continue a movement at SC to rectify this problem. The movement has already begun by requiring, for the first time, an Artifact Description appendix for SC17 papers to win best paper in the technical program. The Artifact Description appendix contains information on how to obtain, compile, and run the software used to get the results in the corresponding paper.
For this challenge, you will take on the role of reviewing an SC16 paper that contains an Artifact Description appendix to see if its results are replicable. We use the ACM definition of replicable:
- For computational experiments, this means that an independent
group can obtain the same result using the author's own
artifacts.—https://www.acm.org/publications/policies/artifact-review-badging
The paper you will review is entitled "The Vectorization of the Tersoff Multi-Body Potential: An Exercise in Performance Portability" by Höhnerbach, Ismail, and Bientinesi. You can go to http://dl.acm.org/citation.cfm?id=3014914 to view the paper.
This paper was chosen from nine Supercomputing conference papers from 2016 that submitted an Artifact Description appendix. The work from this paper adds a high performance implementation of the Tersoff potential to the widely used LAMMPS molecular dynamics (MD) code. Molecular dynamics simulations track the trajectory of up to billions of particles at a time. For most MD simulations, the interaction between particles is described by pairwise potentials such as the Coulomb potential. However, the Tersoff potential is described by a multi-body potential requiring much more computational work.
The challenge will consist of two parts. One part will consist of a short interview with each team as in the other challenges. The other part will consist of a report describing your attempt to replicate the results from the paper. The report must be written in English, using a page size of US Letter, in PDF format, with 1 inch margins, and using a 12 point font size. You will also need to be able to create graphs and tables similar to those from the paper.
Most importantly, in the report, you will need to explain how you got your results and compare your results to the results of the authors. Your scores will not depend on whether your results match the results in the paper. The goal of the report is to state if you can or cannot replicate the results in the paper. If you cannot replicate the paper's results, you can still obtain full points by presenting your data and explaining why you cannot replicate the paper's results.
As a last note, you will only be required to report results for one compute architecture. If your cluster contains both a CPU and an accelerator, results for either one may be used for this part of the competition.
- For computational experiments, this means that an independent
group can obtain the same result using the author's own
- MrBayes
https://github.com/NBISweden/MrBayes
Cassava is the main staple food for 800 million people globally. Currently, about half of the world production of cassava is in Africa and many African families eat cassava for breakfast, lunch and dinner. Thus, cassava holds great promise for feeding Africa's growing population and reducing rural and urban poverty.
In east Africa, Cassava production is being affected by two devastating viruses—Cassava brown streak virus (CBSV) and Uganda cassava brown streak virus (UCBSV), with up to 100% yield loss for smallholder farmers in the region. These viruses are transmitted by 34 species of whiteflies called Bemisia tabaci. They have the same physical traits, making them impossible to identify without genomic sequencing. Using supercomputing, genomic and evolution history, Dr. Laura Boykin is leading this effort, funded by the Bill and Melinda Gates Foundation, to identify the species of whiteflies that transmit these viruses, to prevent this crop devastation. Her team is gathering big volume of useful data, now publicly available via WhiteFlyBase, which will help researchers breed new strains of cassava that resist the whitefly.
MrBayes is one of the main tools that Dr. Boykin used in identifying the species of whiteflies. It is a program for Bayesian phylogenetic inference and model choice across a wide range of phylogenetic and evolutionary models. MrBayes uses Markov chain Monte Carlo (MCMC) methods to estimate the posterior distribution of model parameters. Bayesian approaches were introduced into phylogenetics in the mid-1990s and have enjoyed enormous popularity since then, including among entomologists.[1] For some background of the species identification process students can view the paper by Dr Laura Boykin and Dr Paul De Barro. [2]
MrBayes coupled with the Beagle library will give HPC administrators plenty of options to optimise the utilisation of their clusters.[3] Students will be required to research and choose the most efficient installation type of MrBayes for their own cluster, design a run which will complete within the given timeframe and produce an accurate consensus tree with good posterior probability.
Practice datasets for this application: https://github.com/anders-savill/mrbayes-scc17/blob/master/README.md
[1] Bayesian Phylogenetics and Its Influence on Insect Systematics Annual Review of Entomology. Fredrik Ronquist and Andrew R. Deans
[2] A practical guide to identifying members of the Bemisia tabaci species complex: and other morphologically identical species. Laura M. Boykin and Paul J. De Barro.
[3] MrBayes 3.2: Efficient Bayesian Phylogenetic Inference and Model Choice Across a Large Model Space. Fredrik Ronquist, Maxim Teslenko, Paul van der Mark, Daniel L. Ayres, Aaron Darling, Sebastian Höhna, Bret Larget, Liang Liu, Marc A. Suchard, and John P. Huelsenbeck.
- Born
Instructions
Born is a seismic imaging tool for constructing subsurface images using sound waves. These techniques are the industry standard for understanding subsurface geology. Such an analysis may help geologists identify oil and gas reservoirs hidden beneath the earth's surface. Thousands of ultrasensitive receivers are laid out on the earth to record the sound waves as they echo within the earth. A single seismic experiment involves broadcasting sound energy through the subsurface and recording a signal. Some of the energy transmitted into the earth reflects due to differences in subsurface properties (e.g. density, composition) as a function of position. A seismic survey involves conducting thousands of different experiments varying the positions of the source and/or receivers.
Reverse Time Migration (RTM) is one process to create an image of the subsurface in terms of position (x,y,z) from the seismic data which is a function of source and receiver position and time.
RTM involves conducting two computational experiments. In both cases, the wave equation is used to propagate a wavefield in time using different forcing functions. In the first phase, the source signature is injected into the earth and propagated forward in time. The second phase uses the data recorded at the receivers as the forcing function and propagates backward in time. A final image of the subsurface is updated by multiplying the source and receiver wavefields at every x,y,z.
- Mystery Application
At the start of the competition, teams will be given an application and datasets for a mystery application. Students will be expected to build, optimize and run this mystery application all at the competition.
- Cloud Component
To meet the ever growing demand for access to research computing, more and more companies and institutions are migrating high-performance and high-throughput computing workloads to the cloud. Easy access to utility computing resources is becoming essential, especially in industry, and these resources fill an important niche in the HPC ecosystem. For 2017, the Student Cluster Competition (SCC) will be exploring what it means to use cloud workflows alongside traditional HPC clusters.
This year, in addition to the physical competition clusters residing on the competition floor, teams will have access to a cloud HPC environment to run SCC applications. The cloud HPC environment will be hosted by Cycle Computing on Microsoft Azure and will utilize Cycle Computing’s CycleCloud software to manage and provision the environment. Teams will have control over the software installed on the cluster and will be able to choose whatever cloud instances are available in the Microsoft Azure public cloud.
While resource constraints for the physical competition clusters on the floor revolve around power and space utilization, the cloud portion of the competition will be constrained by a simulated monetary budget. Teams will be allocated two budgets by the SCC: one for a sandbox environment leading up to the competition and another for an environment allocated during the competition itself. The CycleCloud software will help teams estimate the "burn rate" of their cloud clusters and can be configured to send alerts when the team starts to get close to their budgetary limits.
To provide an even playing field in the cloud environment, Cycle Computing and the SCC committee will place controls around the cloud environment similar to how the physical clusters are constrained on the competition floor. These controls will include:
- Cost controls to prevent teams from spending more than the allocated amount.
- Network and firewall rules in the Cloud Service Provider (CSP) account to prevent unauthorized outbound and inbound network access during the competition.
- Role Based Access Controls (RBAC) to prevent teams from altering the CycleCloud software to work with other cloud accounts.
Prior to the competition, teams are expected to be learning and testing applications on their physical clusters as well as on cloud HPC resources. Teams will be provided access to a cloud sandbox or test environment so they can familiarize themselves with the CycleCloud software and Azure machine types to determine which cluster architecture is best suited to their workload.
During the competition, teams may run any and all applications using their physical cluster on the competition floor and/or using their cloud cluster resources as long as they do not exceed power constraints for their physical clusters and they do not exceed the allocated budget for their cloud clusters. Teams are not required to utilize the cloud environment and will not be graded on it’s use. However, it is highly recommended that teams take advantage of these cloud resources to complete more datasets using both their physical clusters and the extra cluster resources in the cloud.
These extra resources will not change the rules of the competition for the physical competition clusters, but the computing accomplished in the cloud will be considered in each team's scores.
Sandbox accounts will be sent out to each team by July 28th.
- Cost controls to prevent teams from spending more than the allocated amount.
- Power Shutoff Activity
Some time during the 45 and 1/2 hours of the general competition the power will be shut-off at least once. The exact timing of the shutdown(s) are secret and may happen day or night. You and your team will need to know how to bring the hardware and software back from a full unscheduled power outage and how to resume any workload you were processing at that time. This exercise is designed to simulate real world events that system staff must respond to. This activity will allow your team to demonstrate their systems skills by recovering the system.
This has happened before. During the first Student Cluster Competition, in 2007, the power to the Reno Convention Center suddenly failed. The entire show floor went dark. It turned out that the power coming to the convention center was inadequate for Supercomputing's high-performance machines. Power was out for an hour or so, followed by what the press described as "the world's largest reboot". After the conference, crews were seen laying additional power cables across Virginia Street.
Our competition clusters, of course, went down. When the power was restored, some teams, who had been checkpointing their systems, resumed their computations quickly. Other teams, who had not been saving data, lost many hours of work and had to start over. The experience prompted discussions about checkpointing in the real world—the tradeoff between protecting against possible disasters at a cost of reducing computations.
Since power and other failures are the realities of modern computing systems, we would like to encourage cluster teams to understand the tradeoffs, and to consider what is needed in real life. We turn this thought-provoking accident into an activity to capture the think-on-your-feet spirit of the first competition.
Details:
The power will be shut off at the breaker to both monitored circuits for all teams. Once the power is shut off, all teams will be asked to leave their booths, and each booth will be inspected to make sure that everything is powered off. All teams will be let back in their booths at the same time to begin the procedures for recovering from the power failure. The full power-off and restoration logistics will be provided on site before the competition begins.
Some rules to be aware of:
Only the undergraduate team members are allowed to participate in bringing the system back up. No vendor or advisor help is allowed.- The advisor is not allowed to provide technical assistance during the competition, however, he/she is encouraged to run for food and snacks for their team and cheer during the long nights.
Any hardware failures can be swapped based on the rules.- No changes to the physical configuration are permitted after the start of the competition. In the case of hardware failure, replacements can be made while supervised by an SCC committee member.
Battery backups are prohibited in the competition this year.- No battery backup or (Uninterrupted Power Supply) UPS systems are allowed to be used during the competition.
Teams that are not present in their booth will need to restore their clusters once they return.- In this instance, clusters will be unplugged from the SCC PDU by the SCC committee after power is shut down and the cluster will remain unplugged until the team returns.
SCC committee initiated power shutdown will not happen during setup or benchmarking. Please let us know if you have any questions. - Poster Session
The Overall SCC Winner will be the team with the highest score when combining their correctly completed workload of the four competition applications, mystery application, best benchmark run, application interviews, and HPC interview. The HPC interview will take into consideration the team's participation in the SC17 conference as well as their ability to wow the judges on their competition know-how.
Teams will be required to attend other aspects of the convention beyond the Student Cluster Competition, which will be included in their final score. Further details will be provided before the competition.