jobqueue jobqueue is a program for executing jobs in parallel to complete all jobs as fast as possible.
Download

jobqueue Ranking & Summary

jobqueue Tags

queue parallel jobs execute jobs parallel execution queue jobs

jobqueue Description

jobqueue is a program for executing jobs in parallel to complete all jobs as fast as possible. jobqueue is a program for executing jobs in parallel to complete all jobs as fast as possible. The jobqueue project can be used to distribute a set of jobs to a specific number of processors and/or machines. New jobs are started as older jobs finish.SYNTAX: jobqueue jobqueue is a tool for executing lists of jobs on several processors or machines in parallel. jobqueue reads jobs (shell commands) from files. If no files are given, jobqueue reads jobs from stdin. Each job is executed in a shell environment (man 3 system).-n x / --nodes=x, jobqueue keeps at most x jobs running in parallel. jobqueue issues new jobs as older jobs are finished.-e / --execution-place, each job is executed by passing an execution place id as a parameter. The execution place defines a virtual execution place for the job, which can be used to determine a machine to execute the job. The place id is an integer from 1 to x (given with -n). If command "foo" is executed from a job list, jobqueue executes "foo x", where x is the execution place id.-m list / --machine-list list, read contents of list file, and count each non-empty and non-comment line to be an execution place. Pass execution place for each executed job as a parameter. The difference to -e is that -e passes the execution place as an integer, but this option passes the execution place as a name for the job. Also, this option implies "-n x", where x is the number of names read from the file.-v / --verbose, enter verbose mode. Print each command that is executed.--version, print version numberEXAMPLE 1: A file named MACHINES contains a list of machines to processjobs from a job file named JOBS. Each line in the JOBS file follows thesame pattern:./myscript data0./myscript data1./myscript data2...MACHINES file contains 4 machines:machine0machine1machine2machine3./myscript might do something like this:#!/bin/bash# This is the dataX parameter from JOBS filedata="$1"# determine the machine that will execute this jobmachine=$2echo $machine $datassh $machine remotecommand $dataTo execute jobs on the machines, issue:jobqueue -m MACHINES JOBSExecution will print something like this:machine0 data0machine1 data1machine2 data2machine1 data4machine3 data3machine0 data5machine1 data7machine2 data6All jobs done (8)EXAMPLE 2: run echo 5 times printing the execution place each time for i in $(seq 5) ; do echo echo ; done |jobqueue -n2 -eprints something like "1 2 1 2 1".What's New in This Release:· Machine list (-m) can now be used to specify the maximum number of simultaneous jobs for each machine.

jobqueue Related Software