Running on NEMO

It is also possible to submit jobs to the NEMO cluster. The jobs will not run directly on the NEMO machines, but will be redirected to virtual machines, which provide (almost) the same setup which you are used to from the normal ATLAS-BFG worker nodes (more details below).

Submitting jobs

To submit Jobs to NEMO with slurm, use the partition nemo_vm_<group-id> using the sbatch option -p or --partition with the argument nemo_vm_<group-id>.

The <group-id> depends on which group you belong to. Check the following table to find out <group-id>.

Group	`<group-id>`
Herten	`atlher`
Heinemann	`atlhei`
Jakobs/Weiser	`atljak`
Schumacher	`atlsch`

For example, if you belong to the atlsch group, submit a script with

sbatch -p nemo_vm_atlsch <script>

Submitting jobs to this partition will result in the startup of virtual machines according to the needs of the queue. The VMs are 40-core machines with a total memory of 100 GB. Whenever there are already VMs running with empty slots at the time of submission, your jobs will land there. Machines are usually drained after 24 hours, or after they are idle for ~15 minutes.

The startup time is about 2 minutes. If squeue gives you the reason PENDING (Resources), this can be due to the fact that there is a queue on NEMO. You can check the usage here: http://www.hpc.uni-freiburg.de/nemo/usage (figure "NEMO Usage Statistics").