Overview
Task Spooler (tsp) is not installed on every Linux server. However, we mention it here because if it happens to be available, it is a straightforward way to run many jobs.
Task Spooler is a type of job queuing software. There are many more complicated job queueing software packages. Task Spooler is a particularly simple tool. It only runs on one server for one user.
The tsp command will run a job in the background; this lets you run other commands while tsp is busy running your program. The tsp command will even keep running your jobs after you logout or disconnect from the server.
In its normal setup, only one job will run with tsp at a time. Once the first job finishes, tsp will automatically start the next job. This list of waiting jobs is called queuing, and is analogous to the lineup at a grocery store.
Let’s start taking a more concrete look at how to use the tsp command.
Running Jobs
Just put tsp in front of the command you want to run. Note that tsp can only directly run a single command, but itself does not handle the special functionality of the Unix shell. So for example if you use a command containing pipes or output redirects, then it is easiest to put those shell commands into a file. This file is called a shell script, but is out of the scope of our current discussion.
Let’s run a fictitious program called `run_simulation` using tsp:
tsp ./run_simulation -t 10 tsp ./run_simulation -t 3 tsp ./run_simulation -t 20 tsp ID State Output E-Level Times(r/u/s) Command [run=1/1] 0 running /tmp/ts-out.Smoz2i ./run_simulation -t 10 1 queued (file) ./run_simulation -t 3 2 queued (file) ./run_simulation -t 20
We have queued up 3 jobs, each running a run_simulation. We’ll move on now to ways of managing jobs and getting output.
Listing Jobs
You can view your list of tsp jobs:
tsp ID State Output E-Level Times(r/u/s) Command [run=1/1] 1 running /tmp/ts-out.IiCgZe ./run_simulation -t 10 2 queued (file) ./run_simulation -t 10 0 finished /tmp/ts-out.QUFZ40 0 2.00/0.00/0.00 ./run_simulation -t 2
The number from the ID column will be used to refer to particular jobs. The State column tells you the progress tsp is making in your job queue.
One important caveat – the list of jobs is kept in memory. So if your server reboots then you will lose any existing jobs. So it is important to collect any needed information about completed jobs.
Inspecting Jobs
Let’s assume you have some finished tsp jobs that look like this:
tsp ID State Output E-Level Times(r/u/s) Command [run=1/1] 0 finished /tmp/ts-out.Smoz2i 0 10.04/0.00/0.00 ./run_simulation -t 10 1 finished /tmp/ts-out.1UDMiN 0 3.02/0.00/0.00 ./run_simulation -t 3 2 finished /tmp/ts-out.twnIHU 0 20.11/0.00/0.00 ./run_simulation -t 20
Note that tsp gives each job an ID. You can see the output of job with ID 1:
tsp -c 1
This is sample output from
a program!
And to see detailed information about the job:
tsp -i 1 Exit status: died with exit code 0 Command: ./run_simulation -t 3 Slots required: 1 Enqueue time: Fri Jul 14 10:45:28 2017 Start time: Fri Jul 14 10:45:38 2017 End time: Fri Jul 14 10:45:41 2017 Time run: 3.021296s
Cleaning up Jobs
To remove a queued job with ID 2:
tsp -r 2
To clear the list of all finished jobs:
tsp -C
To kill (stop) a running job with ID 2:
tsp -k 2
Concurrent Jobs
The tsp command can run more than one job at the same time. In the tsp queue list, the header indicates how many jobs are running, and how many are allowed to run.
However, if your job already uses multiple cores or if your job uses most of the server’s RAM, then you should not run multiple tsp jobs at the same time.
For example, to run 4 jobs at the same time:
tsp ID State Output E-Level Times(r/u/s) Command [run=0/1] tsp -S 4 tsp ID State Output E-Level Times(r/u/s) Command [run=0/4]