Scheduling system

In its default configuration, Lynx will run MOAB/Torque job scheduler.

We recommend passing the options to qsub in a batch script file. Include these options in your script:

• -N Job name
• -q Queue name
• -n Number of processors
• -o Output file name
• -e Error file name
• -l Resource_list

Mpi jobs

Here is a batch script for an MPI program requesting 60 processors on 5 nodes for 1 hour in the regular queue.

#!/bin/bash
#PBS -N MpiJob              
#PBS -q regular                 
#PBS -l mppwidth=60          
#PBS -l mppnppn=12              
#PBS -l walltime=01:00:00      
#PBS -e Myjob.out             
#PBS -j eo                     
#PBS -V                        
cd $PBS_O_WORKDIR               
aprun -n 60 ./mpi-job.exe         

OpenMp jobs

Here is a batch script for an OpenMp program requesting 1 processor with 6 OMP threads for 1 hour in the regular queue.

#!/bin/bash
#PBS -N OMPJob              
#PBS -q regular  
#PBS -l mppwidth=1
#PBS -l mppdepth=6
#PBS -l walltime=01:00:00
#PBS -e Myjob.out
#PBS -j oe
#PBS -V
cd $PBS_O_WORKDIR
export OMP_NUM_THREADS=6
aprun -n 1 -d 6 ./omp-job

Scheduling policy

To ensure that Lynx is available to users for quick testing and debugging jobs during the day, a "shortpool" reservation has been activated on the Lynx system. With the shortpool, 30 compute nodes are reserved for quick-turnaround jobs (1 hour or shorter) from 9 a.m. to 4 p.m. Monday through Friday. The shortpool reservation is enforced by the scheduler and permitted jobs will automatically use the shortpool nodes. We may change the number of shortpool nodes in the future, depending on utilization and user feedback.

The wallclock limit at other times and on non-shortpool nodes during the day remains 12 hours. However, because of the shortpool, users will experience better turnaround with shorter and smaller jobs during daytime hours, even on the unreserved nodes.

If you have any questions or concerns about the setting, please send email to cislhelp@ucar.edu.

Submitting and managing jobs

List queues

To list all queues available for job execution, use the command qstat with the -Q option.

qstat -Q

Submit jobs

To submit a job for execution, use the command qsub and the name of your batch script.

qsub script

Monitor jobs in the queue

To show information about your jobs, use the command qstat or showq.

Use "-u" to specify a user group. The following command displays the information of your current jobs.

qstat -u username

Delete jobs in the queue

To stop and remove a job from the queue, use the command qdel.

qdel jobid

The job ID is produced by qsub and can be obtained through qstat or showq.

More details of qdel can be found in the qdel man page.