Changing Job Attributes
It is possible to modify the attributes of a submitted job using the
scontrol command while the job is running or pending in the queue. However, some job attributes cannot be updated during runtime like GPU(s), node(s), and memory allocations.
Only a few job attributes can be changed after a job is submitted. These attributes include:
- wall clock limit
- job name
- job dependency
For some of the cases, these attributes can only be updated for the pending jobs.
See some of the examples here :
To increase a job’s time limit as it approaches its initial submission deadline.
scontrol update JobId=$JobID timelimit=<new timelimit>
The format set is Minutes, Minutes: Seconds, Hours:Minutes: Seconds, Days-Hours, Days-Hours: Minutes, or Days-Hours:Minutes: Seconds.
To change the job dependency:
scontrol update JobId=$JobID_1 dependency=afterany:$JobID_2
A User can prevent a pending job from being started by placing one’s jobs in a ‘hold’ state by using
scontrol hold <job_id>
To release a queued job from the ‘hold’ state :
scontrol release <job_id>
To cancel and rerun (requeue) a particular job:
scontrol requeue <job_id>
To cancel either running or pending jobs:
To cancel all jobs owned by a user
scancel -u <user>