Difference between revisions of "Slurm Services"
From wiki.hpc.mk
(Created page with "Documentation for creating SLURM Workload Manager") |
|||
| Line 1: | Line 1: | ||
Documentation for creating SLURM Workload Manager | {|class="wikitable" style="width: 100%; background-color:#ffffff; border-width: 0px" | ||
!style="text-align:left; background-color:#F1EDEC"|Contents | |||
|- | |||
| | |||
#[[Slurm_Services#Services_Documenting|Documentation for creating SLURM Workload Manager]] | |||
#[[Slurm_Services#Slurm_Example|Example by executing a simple script]] | |||
|} | |||
__NOTOC__ | |||
'''<h1 id="Services_Documenting">Documentation for creating SLURM Workload Manager</h1>''' | |||
Creating a scheduler in a heterogeneous cluster environment covers the following: First it is necessary to define which will be the main (master) node that will properly forward the user-defined scripts in the form of jobs to the defined machines in the cluster. There are several different platforms for creating a cluster environment, in this document SLURM Workload Manager will be discussed. | |||
Advantages of using Slurm as a task environment are: | |||
* Support for high cluster systems and multiprocessor tasks - The SLURM environment enables the start-up, execution and monitoring of parallel tasks implemented via Message Passing Interface (MPI), on part of the allocated nodes as well as allowing efficient use of resources (nodes) according to a specific policy users, | |||
* Task profiling - Periodically review each resource assigned to a specific task (CPU runtime, RAM, power consumption, network resources, and disk space usage), | |||
* Support for MapReduce + algorithm, | |||
* Support for creating a sequence of tasks, ie one task can be divided into several sub-tasks that are performed in parallel for more efficient use of the given resources, | |||
* Database integration - where all user parameters and settings are stored, | |||
* Use of graphic resources to perform tasks - A large number of optional possibilities for additional use of graphic resources given to a specific task / tasks in order to better perform advanced algorithms in the field of machine learning. | |||
Revision as of 10:10, 31 August 2021
| Contents |
|---|
Documentation for creating SLURM Workload Manager
Creating a scheduler in a heterogeneous cluster environment covers the following: First it is necessary to define which will be the main (master) node that will properly forward the user-defined scripts in the form of jobs to the defined machines in the cluster. There are several different platforms for creating a cluster environment, in this document SLURM Workload Manager will be discussed. Advantages of using Slurm as a task environment are:
- Support for high cluster systems and multiprocessor tasks - The SLURM environment enables the start-up, execution and monitoring of parallel tasks implemented via Message Passing Interface (MPI), on part of the allocated nodes as well as allowing efficient use of resources (nodes) according to a specific policy users,
- Task profiling - Periodically review each resource assigned to a specific task (CPU runtime, RAM, power consumption, network resources, and disk space usage),
- Support for MapReduce + algorithm,
- Support for creating a sequence of tasks, ie one task can be divided into several sub-tasks that are performed in parallel for more efficient use of the given resources,
- Database integration - where all user parameters and settings are stored,
- Use of graphic resources to perform tasks - A large number of optional possibilities for additional use of graphic resources given to a specific task / tasks in order to better perform advanced algorithms in the field of machine learning.
