This week, we’ll start with an introduction to OSC and supercomputers. Then, we’ll see how we can submit shell scripts to OSC’s queue for compute jobs with the widely-used SLURM resource manager. Finally, we will learn about installing and managing software with Conda, and loading pre-installed software with the “module” system.
Some of the things you will learn this week:
When and why we need to run our analyses on supercomputers.
Key terminology around supercomputers.
Different ways to access and transfer data to and from OSC.
Different ways to start “compute jobs”: via OnDemand, with interactive jobs, and with scripts.
Some strategies around requesting appropriate resources for your compute jobs.
The SLURM/sbatch syntax to request specific resources for your compute jobs.
How to monitor SLURM jobs.
How to load pre-installed software at OSC with the “module” system.
How to install and manage software with Conda.
This week’s required reading is a little unusual, as it is not a book chapter. Instead, you will read part of the OSC documentation. There are some pointers below, but you are welcome to use your own judgment in how much you want to read. You are also encouraged to simply look around on OSC’s website a bit to get an idea of what’s there, and bookmark pages that you think will be useful for you later on.
Part of the OSC documentation:
Your first starting point should be the New User Resource Guide At least read or carefully look at the first few pages including “HPC Basics” and “Getting Connected”.
Your second starting point should be the Batch Processing at OSC pages Again, at least read or carefully look at the first few pages including “Batch System Concepts”, “Batch Execution Environment”, and “Job Scripts” (in that last section, you can stop when you reach “Considerations for parallel jobs”, unless this is something that interests you).
Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".