Summary and Schedule
Prerequisite
Before joining Reproducible HPC workflows using JUBE, participants should be able to
- login to an HPC system and navigate a UNIX shell.
- edit text files with an editor available on the HPC system.
- submit jobs to a job scheduler present on the HPC system.
- basic shell, Python, or Perl programming knowledge.
- basic understanding of regular expressions.
Summary
This course provides an introduction into the Jülich Benchmarking Environment (JUBE), a workflow management system created for running performance benchmarks on high-performance computing (HPC) systems. JUBE can also be used for general workflows involving running applications on HPC systems, and this course will further provide an example on how to run the molecular-dynamic code GROMACS and will discuss aspects of workflow reproducibility in JUBE.
🚧 Under Construction 🚧
This lesson is still under construction and will likely change abrubtly and frequently—please be patient.
If you are looking for an introduction to the Jülich Benchmarking Environment (JUBE) in the meantime, feel free to head over to the JUBE tutorial pages of the Jülich Supercomputing Centre.
This is a lesson built with The Carpentries Workbench.
| Setup Instructions | Download files required for the lesson | |
| Duration: 00h 00m | 1. Introduction |
How can I run an HPC workflow reproducibly? What is the easiest way to create a scaling experiment for an HPC application? How do I identify the optimal runtime configuration for an application on an HPC system? |
| Duration: 00h 20m | 2. An example application |
What is a typical HPC application? What is a typical workflow scenario with JUBE? |
| Duration: 00h 35m | 3. Working with JUBE |
Which actions need to be taken when running JUBE? Which commands correspond to which state in a JUBE workflow? |
| Duration: 00h 55m | 4. An initial workflow specification | How to use JUBE to automate the build process of a given HPC application? |
| Duration: 01h 07m | 5. Influencing parameters with tags |
How can a workflow be influenced from the command line without changing
the workflow specification files? |
| Duration: 01h 19m | 6. Spanning parameter spaces |
How to test multiple parameters in a single workflow? How to control multi-dimensional parameter spaces? |
| Duration: 01h 39m | 7. Using templates |
How do I generate dynamic text-based input files for my
workflow. How do I create batch scripts for by workflow. ::::::::::::::::::::::::: |
| Duration: 01h 49m | 8. Including external data |
How to better structure your workflow files to allow for reuse? How to reduce redundancy of definitions? |
| Duration: 02h 04m | 9. Running an application | How can you use JUBE to submit batch jobs on an HPC System? |
| Duration: 02h 19m | 10. Output analysis and presentation |
How can you retrieve output from your HPC application and
workflow? How to generate tables and CSV output from JUBE workflow parameters? |
| Duration: 02h 39m | 11. Thoughts on reproducibility | How can I improve reproducibility of my JUBE workflow? |
| Duration: 02h 59m | 12. Wrap-up and outlook | Which features of JUBE were not covered in this course? |
| Duration: 03h 11m | Finish |
The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.
This lessons focuses on execution of workflows on HPC systems, including interaction with its job scheduler. You will therefore have to
- have access to a system with a job scheduler (e.g., SLURM), and
- setup the JUBE software environment on that system.
Data Sets
This lesson uses the GROMACS software as an example HPC code. GROMACS is a free and open-source software suite for high-performance molecular dynamics and output analysis.
Eventually, downloading GROMACS will be part of the overall HPC workflow, but you can download the 2024.5 version of GROMACS from the download page of GROMACS.
Data sets for this lesson are packaged with this lesson and can be downloaded in preparation.
Software Setup
Software setup can take a significant amount of time, especially when users are not familiar with software setup with Conda or Python virtual environments.
It is best practice to have a special session for software setup prior to the actual workshop.