A cluster is a group of computers connected together to act as one.

Clusters are formed of nodes, each usually has several processors and 10s or hundreds of gigabytes of RAM.

HPCW/SCW have clusters for researchers at Welsh universities to use

ssh login.hpcwales.co.uk for the login node, from there connect to the cluster of your choice

hpcwhosts command shows a list of clusters

Scratch and home are per site, no common storage between sites.

Scratch is faster and has no quotas, its not backed up. home is slower, smaller but backed up

Interactive jobs let you test out the behaviour of a command, but aren’t pratical for running lots of jobs

Batch jobs are suited for submitting a job to run without user interaction.

Job arrays are useful to submit lots of jobs.

Slurm lets you set parameters about how many processors or nodes are allocated, how much memory or how long the job can run.

Slurm can email you when a job starts or finishes.

A lot of software is only available by loading extra modules. This helps prevent problems where two packages are incompatible.

Python3 is one such package.

If you want to install pip packages use the –user option to store the packages in your home directory and load the http-proxy module first.

GNU Parallel lets a single Slurm job start multiple subprocesses

This helps to use all the CPUs on a node effectively.