Week 7: Slurm batch jobs at OSC
1 Overview
As mentioned last week, when you learned about shell scripts:
The end goal is to be able to submit shell scripts as “batch jobs” at OSC, which e.g. allows you to run them simultaneously many times! This is extremely useful because with omics analysis, it’s common to have to run the same step for many samples in parallel.
This week, we’ll cover the remaining piece in being able to run bioinformatics tools efficiently at OSC and beyond: how to submit your shell scripts as batch jobs.
2 Learning goals
Slurm batch jobs – part I
- Different ways to start “compute jobs”: via OnDemand, with interactive jobs, and with Slurm batch jobs
- Strategies around requesting appropriate resources for your compute jobs
- Slurm commands like
sbatchto submit, monitor and manage batch jobs - How to use Slurm options to request specific resources for your compute jobs
Slurm batch jobs – part II
- The most common Slurm options, including those to request:
- More time, cores, and memory for jobs
- An email notification when jobs e.g. fail or finish
- How to run Slurm jobs in practice, including how to submit many jobs at the same time
- How to make sure that your jobs succeeded