Partitions, QoS & Preemption
Each partition (or "pool") has several QoS tiers that determine job priority and preemption behavior. To see which partitions, accounts and QoS you can use, run check_my_partitions on the login node (see Finding Your Account and Partition).
QoS types
| QoS | Purpose | Preempts | Can be preempted by |
|---|---|---|---|
Share-type QoS (e.g. 0.125_48c_8g, 0.75_48c_8g) |
For multi-owner pools; defines each owner's guaranteed slice (CPU/GPU portion). | owner, public |
-- |
owner |
Used on your lab's pool to run above your guaranteed slice (higher than public). |
public |
Share-type QoS |
public (partition: power-general-shared-pool) |
Used on cluster-wide shared pools for friendly or opportunistic runs. | -- | owner, share-type QoS |
public (partition: power-general-public-pool) |
Used on a small, cluster-wide shared group of nodes. Not preemptable. | -- | -- |
Preemption rules
Billing and priority
Each user has a "billing" value, calculated from the amount of resources they request. When you ask for more memory, your billing increases.
Billing affects priority: users who asked for fewer resources in the past get higher priority in the future.
Submitting with the correct QoS
Below are examples of how to use each QoS tier with your account.
Owner QoS (on your lab's pool):
sbatch -A UIDHERE-users_v2 -p UIDHERE-pool --qos=owner --time=02:00:00 run.sh
sbatch -A UIDHERE-users_v2 -p gpu-LAB1-LAB2-pool --qos=0.125_48c_8g --gres=gpu:A100:1 run.sh
Public QoS (friendly, cluster-wide):
sbatch -A public-users_v2 -p power-general-shared-pool --qos=public --time=01:00:00 run.sh
Small, protected CPU pool:
sbatch -A public-users_v2 -p power-general-public-pool --qos=public --time=01:00:00 run.sh
Handy checks
Monitor your jobs and see their QoS and pending reasons:
squeue --me -O "JOBID,ACCOUNT,PARTITION,QOS,STATE,REASON"
sprio -w
If your job was preempted, check:
sacct -j <jobid> --format=JobID,State,Reason