Running Jobs on the Bauer Core Linux Cluster

Once you have an account on the cluster and know how to log in and transfer your files, you are ready to run jobs.

Submitting jobs

Rather than running a job from the command line (by typing something like “blastall -p blastp -i infile -o outfile -d nr”), you need to submit a job to the LSF queueing system. The system will wait until a processor is available to run your job. It will run the job on a different computer (not portal) and return the results to you as if you had run them on portal. This may sound complicated, but all you need to do to submit a job is type the word "bsub" before the command you were going to type anyway. For example, "bsub blastall -p blastp -i infile -o outfile -d nr".

Queues

In order to ensure fair usage of the cluster, there are a number of queues (waiting lines) for jobs. When you submit a job with the bsub command, it will by default go into the normal queue. Alternatively, you can specify a queue: "bsub -q long bsub blastall -p blastp -i infile -o outfile -d nr".

Most queues have a time limit. Any job running for longer than the queue’s time limit will be killed automatically by LSF.It is usually a good idea to run a small part of your job and see how long it takes, in order to figure out to which queue you should submit your job. Also, some queues can be used by only some users. "bqueues -ufred" will say which queues user fred can use. Finally, each queue has a priority, so it may be able to suspend jobs in some other queues. (On average, a shorter job will interrupt a longer one; when the shorter job is done, the longer one will resume.)

Here is a summary of queues that are available.This information (and more) can also be accessed with "bqueues -l".

Queue Name	Description	Suspend jobs in which queues?	Max Job Length	Accessibility
short	High-priority queue	normal, long	1 hour	All
normal	Medium-priority queue. This is the default queue.	long	24 hours	All
long	Low-priority queue	N/A	No limit	All
interactive	For interactive jobs, e.g. graphical programs	normal, long	12 hours	All
CGRshort	High-priority Bauer queue	CGRnormal, CGRlong, short, normal, long	1 hour	Bauer members only
CGRnormal	Medium-priority Bauer queue	CGRlong, normal, long	1 day	Bauer members only
CGRlong	Low-priority Bauer queue	N/A	No limit	Bauer members only

Other LSF commands

For more information on bsub, bqueues, and other LSF commands, read the LSF cheat sheet.

Bauer Cluster Support: cluster_admin@cgr.harvard.edu