Gang scheduling

Algorithm
In computer science, gang scheduling is a scheduling algorithm for parallel systems that schedules related threads or processes to run simultaneously on different processors. Usually these will be threads all belonging to the same process, but they may also be from different processes, where the processes could have a producer consumer relationship or come from the same MPI program. Gang scheduling is used to ensure that if two or more threads or processes communicate with each other, they will all be ready to communicate at the same time. If they were not gang scheduled, then one could wait to send or receive a message to another while it is sleeping, and vice versa. When processors are over subscribed and gang scheduling is not used within a group of processes or threads which communicate with each other, each communication event could suffer the overhead of a context switch. Gang scheduling is based on a data structure called the Ousterhout matrix. In this matrix each row represents a time slice, and each column a processor. The threads or processes of each job are packed into a single row of the matrix. During execution, coordinated context switching is performed across all nodes to switch from the processes in one row to those in the next row. Gang scheduling is stricter than coscheduling. It requires all threads of the same process to run concurrently, while coscheduling allows for fragments, which are sets of threads that do not run concurrently with the rest of the gang. Gang scheduling was implemented and used in production mode on several parallel machines, most notably the Connection Machine CM 5.

In gang scheduling, one to one mapping happens, which means each task will be mapped to a processor. Usually, jobs are considered as independent gangs, but with a bag of gangs scheme, all the gangs can be combined and sent together to the system. When jobs are executed in the system, the execution can never be completed until and unless all the gangs that belong to the same BoG complete their executions. The response time is further affected when a priority job arrives. Whenever a priority job arrives at the system, that job will be given priority with respect to all other jobs, even over the ones which are currently being executed on the processors. In this case, when a priority job arrives, the sub gang which is currently executing on the system will be stopped and all the progress that has been made will be lost and need to be redone. This interruption of the job will further delay the total response time of the BoG.

In the above execution scheme, the tasks which correspond to increasing job size are placed in a queue, with the tasks belonging to the largest gang scheduled first, but this method of execution tends to lead to the starvation of resources of smaller jobs and is therefore unfit to be executed in systems where the number of processors is comparatively low. Blocking case: The processors assigned to the interrupted jobs are blocked and cannot execute other jobs in their queue until the jobs from the damaged processors are cleared.

This algorithm is a modified version of the best fit algorithm. In the best fit algorithm, the PEs are allocated in a sequential order, but in this algorithm, the PEs can be inserted from both directions so as to reduce the overlap between different sets of PEs assigned to different jobs. 1. Left right by size. Here, the PEs can be inserted in sequential order and in reverse sequential order based on the size of the job. If the size of the job is small, the PEs are inserted from left to right and if the job is large, the PEs are inserted from right to left. 2. Left right by slots. Unlike the previous algorithm, where the choice was based on the size of the job, here the choice is dependent on the slot. Now, slots are indicated as being filled, i. e. being filled from the left or from the right. The PEs are allocated to the job in the same order. The number of slots on both sides is approximately equal, so when a new slot is opened, the direction is indicated based on the number of slots in both direction.

Both the capacity based and left right based algorithms do not accommodate the load on individual PEs. Load based algorithms take into account the load on the individual PE while tracking the overlap between sets of PEs assigned to different jobs. 1. Minimal maximum load. In this scheme, PEs are sorted based on the load on them that each job will have on the PEs. The availability of the free PEs in the slot determines the capacity of the slot. Suppose that PEs are allocated to a job which has threads, the PE in the load order (last one) will determine the maximum load that any PE can have which is available in the slot. The slot which has minimal maximum load on any PE is chosen and a number of least loaded free PEs are used in the slot. 2. Minimal average load. Unlike the previous scheme, in which slots were chosen based on the minimal maximum load on PE, here slots are chosen based on the average of the load on the least loaded PEs.

In this algorithm the PEs are assigned in clusters, not individually. The PEs are first partitioned into groups that are power of two. Each member of the group will be assigned a controller and when a job of size n arrives, it is assigned to a controller of size 2[lg 2] (the smallest power to 2 that is larger than or equal to n). The controller is assigned by first sorting all the used slots, and then identifying groups of 2[lg 2] contiguous free processors. If a controller has all the PEs free in some of the slots, then only a newly arrived job will be assigned to that controller. Otherwise a new slot is opened.

In all the above mentioned algorithms, the initial placement policy is fixed and jobs are allocated to the PEs based on that. However, this scheme migrates jobs from one set of PEs to another set of PEs, which in turn improves the run fraction of the system.