Difference between revisions of "Slurm Services"
From wiki.hpc.mk
| Line 13: | Line 13: | ||
'''<h1 id="Services_Documenting">Documentation for creating SLURM Workload Manager</h1>''' | '''<h1 id="Services_Documenting">Documentation for creating SLURM Workload Manager</h1>''' | ||
Creating a scheduler in a heterogeneous cluster environment covers the following: First it is necessary to define which will be the main (master) node that will properly forward the user-defined scripts in the form of jobs to the defined machines in the cluster. There are several different platforms for creating a cluster environment, in this document SLURM Workload Manager will be discussed. | Creating a scheduler in a heterogeneous cluster environment covers the following: First it is necessary to define which will be the main (master) node that will properly forward the user-defined scripts in the form of jobs to the defined machines in the cluster. There are several different platforms for creating a cluster environment, in this document SLURM Workload Manager will be discussed. | ||
Advantages of using Slurm as a task environment are: | Advantages of using Slurm as a task environment are: | ||
* Support for high cluster systems and multiprocessor tasks - The SLURM environment enables the start-up, execution and monitoring of parallel tasks implemented via Message Passing Interface (MPI), on part of the allocated nodes as well as allowing efficient use of resources (nodes) according to a specific policy users, | * Support for high cluster systems and multiprocessor tasks - The SLURM environment enables the start-up, execution and monitoring of parallel tasks implemented via Message Passing Interface (MPI), on part of the allocated nodes as well as allowing efficient use of resources (nodes) according to a specific policy users, | ||
Revision as of 10:10, 31 August 2021
| Contents |
|---|
Documentation for creating SLURM Workload Manager
Creating a scheduler in a heterogeneous cluster environment covers the following: First it is necessary to define which will be the main (master) node that will properly forward the user-defined scripts in the form of jobs to the defined machines in the cluster. There are several different platforms for creating a cluster environment, in this document SLURM Workload Manager will be discussed. Advantages of using Slurm as a task environment are:
- Support for high cluster systems and multiprocessor tasks - The SLURM environment enables the start-up, execution and monitoring of parallel tasks implemented via Message Passing Interface (MPI), on part of the allocated nodes as well as allowing efficient use of resources (nodes) according to a specific policy users,
- Task profiling - Periodically review each resource assigned to a specific task (CPU runtime, RAM, power consumption, network resources, and disk space usage),
- Support for MapReduce + algorithm,
- Support for creating a sequence of tasks, ie one task can be divided into several sub-tasks that are performed in parallel for more efficient use of the given resources,
- Database integration - where all user parameters and settings are stored,
- Use of graphic resources to perform tasks - A large number of optional possibilities for additional use of graphic resources given to a specific task / tasks in order to better perform advanced algorithms in the field of machine learning.
