First Warning - Unity Partition Changes

Hello Unity Users, Due to the unprecedented flood of critical vulnerabilities exposed in the Linux kernel, Unity will move to a weekly kernel update schedule. This means it is no longer possible to support 14 day uninterrupted jobs.

To ensure we can patch vulnerabilities quickly, we will reduce the max time limit to 5 days cluster wide. While this is a substantial decrease from 14 days, it’s still significantly longer than peer clusters like AICR (24 hours) and most ACCESS clusters (24-48 hours).

From historic data, we expect this change to impact at most 10% of Unity users. If you need help, please [contact the Unity team] (/contact/community/) to discuss job checkpointing and strategies.

Queues Split Into High VRAM and Normal and GPU Limits Changes

To help efficiently schedule our most contested resource – high VRAM GPUs – we’re splitting the GPU queue into gpu-highvram (>= 40GB VRAM) and gpu (< 40GB VRAM), similarly gpu-preempt-highvram and gpu-preempt.

GPU quota per PI account remains 64 GPUs in gpu and gpu-preempt. GPU quota in the new high VRAM queues is reduced to 16 GPUs simultaneously per PI group and 8 GPUs simultaneously per user.

Limits in GPU priority queues, such as superpod-a100, remain the same.

Do you need large, parallel GPU resources for your work? You may be eligible for an AICR allocation!

CPU Queue Quota Changes

To encourage queue cycling, we’re reducing simultaneous CPU cores in the CPU queue per PI group from 1000 to 512.

Timeline

Now: Change your high-VRAM jobs to gpu-highvram and gpu-preempt-highvram. The new GPU queues are available and have a higher priority than the old queue to incentivize moving jobs over as soon as possible.

Tuesday, October 27: The high VRAM gpus will be removed from the gpu partition. All general GPU queues will move to the 5 day time limit.

Tuesday, November 3: All CPU queues and priority queues will move to the 5 day time limit.