Optimizing the parallelization - Revision history

Huebsch at 10:10, 28 February 2025

2025-02-28T10:10:41Z

← Older revision		Revision as of 10:10, 28 February 2025
Line 34:		Line 34:
	* Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs. Mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].		* Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs. Mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].
	* The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]]. Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points. Keep in mind that {{TAG\|KPAR}} should factorize the number of '''k''' points.		* The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]]. Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points. Keep in mind that {{TAG\|KPAR}} should factorize the number of '''k''' points.
			* For bulk systems with small unit cells ({{TAG\|NBANDS}} is small, NKPTS=no of k points is large), {{TAGO\|NCORE\|1}} and {{TAGO\|KPAR\|NKPTS}} is optimal.
			* For running on GPUs, you may try
			** MPI ranks = no of GPUs
			** {{TAGO\|KPAR\|no of GPUs}} if memory allows
			** [[Combining_MPI_and_OpenMP\|OpenMP threading]] can be quite important for the parts that still run on the host, because usually no of GPUs is rather small.
	* Finally, use the {{TAG\|IMAGES}} tag to split several VASP runs into separate calculations. The limit is dictated by the number of desired calculations.		* Finally, use the {{TAG\|IMAGES}} tag to split several VASP runs into separate calculations. The limit is dictated by the number of desired calculations.

Schlipf: /* Related tags an articles */

2023-10-20T09:59:23Z

Schlipf at 09:59, 20 October 2023

2023-10-20T09:59:07Z

← Older revision		Revision as of 09:59, 20 October 2023
Line 27:		Line 27:
	# '''Create a list of the relevant [[:Category:Parallelization\|parallelization INCAR tags]]''' for the specific calculation. '''Read the documentation for each of the relevant tags''' to understand the limits and reasonable choices.		# '''Create a list of the relevant [[:Category:Parallelization\|parallelization INCAR tags]]''' for the specific calculation. '''Read the documentation for each of the relevant tags''' to understand the limits and reasonable choices.
	# For any calculation involving [[:Category:Electronic minimization\|electronic minimization]], it can be useful to first '''run VASP with {{TAG\|ALGO}}=''None''''' set in the {{FILE\|INCAR}} file. With {{TAG\|ALGO}}=''None'' the [[:Category:Calculation setup\|computational setup]] for the [[:Category:Electronic minimization\|electronic minimization]] is done without actually performing the minimization. For instance, the FFTs are planned, and the irreducible '''k''' points of the first Brillouin zone are constructed. Therefore, some parameters, e.g., the default number of Kohn-Sham orbitals ({{TAG\|NBANDS}}) and the total number of plane waves, are written to the {{FILE\|OUTCAR}} file while using barely any computational time.		# For any calculation involving [[:Category:Electronic minimization\|electronic minimization]], it can be useful to first '''run VASP with {{TAG\|ALGO}}=''None''''' set in the {{FILE\|INCAR}} file. With {{TAG\|ALGO}}=''None'' the [[:Category:Calculation setup\|computational setup]] for the [[:Category:Electronic minimization\|electronic minimization]] is done without actually performing the minimization. For instance, the FFTs are planned, and the irreducible '''k''' points of the first Brillouin zone are constructed. Therefore, some parameters, e.g., the default number of Kohn-Sham orbitals ({{TAG\|NBANDS}}) and the total number of plane waves, are written to the {{FILE\|OUTCAR}} file while using barely any computational time.
	# Combine the information from the documentation and the dry run into a few possible candidates for a reasonable setup. '''Run test calculations''' on a subset of the production run e.g. by reducing the number of steps.		# Combine the information from the documentation and the [[Command-line arguments#--dry-run / -n\|dry run]] into a few possible candidates for a reasonable setup. '''Run test calculations''' on a subset of the production run e.g. by reducing the number of steps.
	# Run the production calculation with the best performing setup.		# Run the production calculation with the best performing setup.

Huebsch: Huebsch moved page Construction:Optimizing the parallelization to Optimizing the parallelization

2022-05-25T09:33:10Z

Huebsch moved page Construction:Optimizing the parallelization to Optimizing the parallelization

← Older revision	Revision as of 09:33, 25 May 2022
(No difference)

Schlipf: /* Optimizing the parallelization */

2022-05-25T06:55:30Z

Optimizing the parallelization

← Older revision		Revision as of 06:55, 25 May 2022
Line 25:		Line 25:
	To optimize the parallelization, follow this recipe:		To optimize the parallelization, follow this recipe:

	# '''Create a list of the relevant [[:Category:Parallelization\|INCAR tags ~~for parallelization~~]]''' for the specific calculation. '''Read the documentation for each of the relevant tags''' to understand the limits and reasonable choices.		# '''Create a list of the relevant [[:Category:Parallelization\|parallelization INCAR tags]]''' for the specific calculation. '''Read the documentation for each of the relevant tags''' to understand the limits and reasonable choices.
	# For any calculation involving [[:Category:Electronic minimization\|electronic minimization]], it can be useful to first '''run VASP with {{TAG\|ALGO}}=''None''''' set in the {{FILE\|INCAR}} file. With {{TAG\|ALGO}}=''None'' the [[:Category:Calculation setup\|computational setup]] for the [[:Category:Electronic minimization\|electronic minimization]] is done without actually performing the minimization. For instance, the FFTs are planned, and the irreducible '''k''' points of the first Brillouin zone are constructed. Therefore, some parameters, e.g., the default number of Kohn-Sham orbitals ({{TAG\|NBANDS}}) and the total number of plane waves, are written to the {{FILE\|OUTCAR}} file while using barely any computational time.		# For any calculation involving [[:Category:Electronic minimization\|electronic minimization]], it can be useful to first '''run VASP with {{TAG\|ALGO}}=''None''''' set in the {{FILE\|INCAR}} file. With {{TAG\|ALGO}}=''None'' the [[:Category:Calculation setup\|computational setup]] for the [[:Category:Electronic minimization\|electronic minimization]] is done without actually performing the minimization. For instance, the FFTs are planned, and the irreducible '''k''' points of the first Brillouin zone are constructed. Therefore, some parameters, e.g., the default number of Kohn-Sham orbitals ({{TAG\|NBANDS}}) and the total number of plane waves, are written to the {{FILE\|OUTCAR}} file while using barely any computational time.
	# Combine the information from the documentation and the dry run into a few possible candidates for a reasonable setup. '''Run test calculations''' on a subset of the production run e.g. by reducing the number of steps.		# Combine the information from the documentation and the dry run into a few possible candidates for a reasonable setup. '''Run test calculations''' on a subset of the production run e.g. by reducing the number of steps.
Line 31:		Line 31:

	For the common case of [[:Category: Electronic minimization\|electronic minimization]] calculations, the following rules of thumb apply:		For the common case of [[:Category: Electronic minimization\|electronic minimization]] calculations, the following rules of thumb apply:
	Aim to set the number of ranks to the default value of {{TAG\|NBANDS}} divided by a small integer.		* Aim to set the number of ranks to the default value of {{TAG\|NBANDS}} divided by a small integer. Note that VASP will increase {{TAG\|NBANDS}} to accommodate the number of ranks.
	Note that VASP will increase {{TAG\|NBANDS}} to accommodate the number of ranks.		* Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs. Mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].
	Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs.		* The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]]. Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points. Keep in mind that {{TAG\|KPAR}} should factorize the number of '''k''' points.
	Mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].		* Finally, use the {{TAG\|IMAGES}} tag to split several VASP runs into separate calculations. The limit is dictated by the number of desired calculations.
	The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]].
	Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points.
	Keep in mind that {{TAG\|KPAR}} should factorize the number of '''k''' points.
	Finally, use the {{TAG\|IMAGES}} tag to split several VASP runs into separate calculations.
	The limit is dictated by the number of desired calculations.

	==Related tags an articles==		==Related tags an articles==

Schlipf: /* Optimizing the parallelization */

2022-05-25T06:49:58Z

Optimizing the parallelization

← Older revision		Revision as of 06:49, 25 May 2022
Line 5:		Line 5:
	==Optimizing the parallelization==		==Optimizing the parallelization==

	~~Basically, for any~~ repetitive ~~task~~, '''a few ~~iteration steps give a good~~ estimate of the performance''' ~~for~~ the full calculation.		For repetitive tasks, '''a few iterations estimate the performance''' of the full calculation very well.
	For example, run only a few electronic or ionic self-consistency steps (without reaching full convergence) ~~and compare~~ various setups ~~for the parallelization~~.		For example, run only a few electronic or ionic self-consistency steps (without reaching full convergence).
			Compare the time various parallelization setups need to perform these few iterations.

	Try to '''get as close as possible to the actual system'''. ~~This includes both~~ the physical system (atoms, cell size, cutoff, ...) ~~as well as~~ the computational hardware (CPUs, interconnect, number of nodes, ...).		Try to '''get as close as possible to the actual system'''.
			Specifically, use the same or a very similar physical system (atoms, cell size, cutoff, ...);
			run on the target computational hardware (CPUs, interconnect, number of nodes, ...).
	If too many parameters are different, the parallel configuration may not be transferable to the production calculation.		If too many parameters are different, the parallel configuration may not be transferable to the production calculation.

	For any calculation involving [[:Category:Electronic minimization\|electronic minimization]], it can be useful to first '''run VASP with {{TAG\|ALGO}}=''None''''' set in the {{FILE\|INCAR}} file. That is because with {{TAG\|ALGO}}=''None'' the [[:Category:Calculation setup\|computational setup]] for the [[:Category:Electronic minimization\|electronic minimization]] is done without actually performing the minimization. For instance, the FFTs are planned, and the irreducible '''k''' points of the first Brillouin zone are constructed. Therefore, some parameters, e.g., the default number of Kohn-Sham orbitals ({{TAG\|NBANDS}}) and the total number of plane waves, are written to the {{FILE\|OUTCAR}} file while using barely any computational time.		In our experience, VASP yields the best performance by '''combining multiple parallelization options''' because the parallel efficiency of each level drops near its limit.
			By default, VASP distributes the number of bands ({{TAG\|NBANDS}}) over the available MPI ranks.
	In our experience, VASP yields the best performance by '''combining multiple parallelization options'''~~. This is~~ because the parallel efficiency of each level drops near its limit.		But it is often beneficial to add parallelization of the FFTs ({{TAG\|NCORE}}), parallelization over '''k''' points ({{TAG\|KPAR}}), and parallelization over separate calculations ({{TAG\|IMAGES}}).
	By default, VASP distributes the number of bands ({{TAG\|NBANDS}}) over the available MPI ranks. But it is often beneficial to add parallelization of the FFTs ({{TAG\|NCORE}}), parallelization over '''k''' points ({{TAG\|KPAR}}), and parallelization over separate calculations ({{TAG\|IMAGES}}). Additionally, there are some parallelization options for specific algorithms in VASP, e.g., {{TAG\|NOMEGAPAR}} for parallelization over imaginary frequency points in <math>GW</math> and RPA calculations. In summary, '''VASP parallelizes with'''		Additionally, there are some parallelization options for specific algorithms in VASP, e.g., {{TAG\|NOMEGAPAR}} for parallelization over imaginary frequency points in <math>GW</math> and RPA calculations. In summary, '''VASP parallelizes with'''

	::<math>		::<math>
Line 20:		Line 23:
	</math>		</math>

	~~First~~, '''~~create~~ a list of the relevant [[Category:Parallelization\|INCAR tags for parallelization]]''' for the specific calculation. ~~Then,~~ '''~~read~~ the documentation for each of the relevant tags''' to ~~find~~ reasonable ~~settings~~ to '''run test calculations'''.		To optimize the parallelization, follow this recipe:

			# '''Create a list of the relevant [[:Category:Parallelization\|INCAR tags for parallelization]]''' for the specific calculation. '''Read the documentation for each of the relevant tags''' to understand the limits and reasonable choices.
			# For any calculation involving [[:Category:Electronic minimization\|electronic minimization]], it can be useful to first '''run VASP with {{TAG\|ALGO}}=''None''''' set in the {{FILE\|INCAR}} file. With {{TAG\|ALGO}}=''None'' the [[:Category:Calculation setup\|computational setup]] for the [[:Category:Electronic minimization\|electronic minimization]] is done without actually performing the minimization. For instance, the FFTs are planned, and the irreducible '''k''' points of the first Brillouin zone are constructed. Therefore, some parameters, e.g., the default number of Kohn-Sham orbitals ({{TAG\|NBANDS}}) and the total number of plane waves, are written to the {{FILE\|OUTCAR}} file while using barely any computational time.
			# Combine the information from the documentation and the dry run into a few possible candidates for a reasonable setup. '''Run test calculations''' on a subset of the production run e.g. by reducing the number of steps.
			# Run the production calculation with the best performing setup.

	For ~~example, in~~ the case of [[:Category:Electronic minimization\|electronic minimization]], ~~aim~~ to set the number of ranks to the default value of {{TAG\|NBANDS}} divided by a small integer. Note that VASP will increase {{TAG\|NBANDS}} to ~~accomodate~~ the number of ranks. Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs~~, but mind~~ that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].		For the common case of [[:Category: Electronic minimization\|electronic minimization]] calculations, the following rules of thumb apply:
			Aim to set the number of ranks to the default value of {{TAG\|NBANDS}} divided by a small integer.
			Note that VASP will increase {{TAG\|NBANDS}} to accommodate the number of ranks.
			Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs.
			Mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].
	The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]].		The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]].
	Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points.		Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points.

Huebsch at 12:20, 12 April 2022

2022-04-12T12:20:01Z

← Older revision		Revision as of 12:20, 12 April 2022
Line 5:		Line 5:
	==Optimizing the parallelization==		==Optimizing the parallelization==

	Basically, a few steps ~~of any repetitive tasks~~ give a good estimate of the performance for the full calculation.		Basically, for any repetitive task, '''a few iteration steps give a good estimate of the performance''' for the full calculation.
	For example, run only a few electronic or ionic self-consistency steps (without reaching full convergence) and compare various setups for the parallelization.		For example, run only a few electronic or ionic self-consistency steps (without reaching full convergence) and compare various setups for the parallelization.

	Try to get as close as possible to the actual system. This includes both the physical system (atoms, cell size, cutoff, ...) as well as the computational hardware (CPUs, interconnect, number of nodes, ...).		Try to '''get as close as possible to the actual system'''. This includes both the physical system (atoms, cell size, cutoff, ...) as well as the computational hardware (CPUs, interconnect, number of nodes, ...).
	If too many parameters are different, the parallel configuration may not be transferable to the production calculation.		If too many parameters are different, the parallel configuration may not be transferable to the production calculation.

	It can be useful to first run VASP with {{TAG\|ALGO}}=''None'' set in the {{FILE\|INCAR}} file. That is because the [[:Category:Calculation setup\|computational setup]] for the [[:Category:Electronic minimization\|electronic minimization]] is done without actually performing the minimization. For instance, the FFTs are planned, and the irreducible '''k''' points of the first Brillouin zone are constructed. Therefore, some parameters, e.g., the default number of Kohn-Sham orbitals ({{TAG\|NBANDS}}) and the total number of plane waves, are written to the {{FILE\|OUTCAR}} file while using barely any computational time.		For any calculation involving [[:Category:Electronic minimization\|electronic minimization]], it can be useful to first '''run VASP with {{TAG\|ALGO}}=''None''''' set in the {{FILE\|INCAR}} file. That is because with {{TAG\|ALGO}}=''None'' the [[:Category:Calculation setup\|computational setup]] for the [[:Category:Electronic minimization\|electronic minimization]] is done without actually performing the minimization. For instance, the FFTs are planned, and the irreducible '''k''' points of the first Brillouin zone are constructed. Therefore, some parameters, e.g., the default number of Kohn-Sham orbitals ({{TAG\|NBANDS}}) and the total number of plane waves, are written to the {{FILE\|OUTCAR}} file while using barely any computational time.

	~~Often~~, VASP yields the best performance by combining multiple parallelization options. This is because the parallel efficiency of each level drops near its limit.		In our experience, VASP yields the best performance by '''combining multiple parallelization options'''. This is because the parallel efficiency of each level drops near its limit.
	By default, VASP distributes the number of bands ({{TAG\|NBANDS}}) over the available MPI ranks. But it is often beneficial to add parallelization of the FFTs ({{TAG\|NCORE}}), parallelization over '''k''' points ({{TAG\|KPAR}}), and parallelization over separate calculations ({{TAG\|IMAGES}}). Additionally, there are some parallelization options for specific algorithms in VASP, e.g., {{TAG\|NOMEGAPAR}} for parallelization over imaginary frequency points in <math>GW</math> and RPA calculations. In summary, VASP parallelizes with		By default, VASP distributes the number of bands ({{TAG\|NBANDS}}) over the available MPI ranks. But it is often beneficial to add parallelization of the FFTs ({{TAG\|NCORE}}), parallelization over '''k''' points ({{TAG\|KPAR}}), and parallelization over separate calculations ({{TAG\|IMAGES}}). Additionally, there are some parallelization options for specific algorithms in VASP, e.g., {{TAG\|NOMEGAPAR}} for parallelization over imaginary frequency points in <math>GW</math> and RPA calculations. In summary, '''VASP parallelizes with'''

	::<math>		::<math>
Line 20:		Line 20:
	</math>		</math>

	First, create a list of the relevant [[Category:Parallelization\|INCAR tags for parallelization]] for the specific calculation. Then, the documentation for each of the relevant ~~[[Category:Parallelization\|INCAR~~ tags ~~for parallelization]] should help~~ to find reasonable settings to run test calculations.		First, '''create a list of the relevant [[Category:Parallelization\|INCAR tags for parallelization]]''' for the specific calculation. Then, '''read the documentation for each of the relevant tags''' to find reasonable settings to '''run test calculations'''.

	For example, in the case of [[:Category:Electronic minimization\|electronic minimization]], aim to set the number of ranks to the default value of {{TAG\|NBANDS}} divided by a small integer. Note that VASP will increase {{TAG\|NBANDS}} to ~~match~~ the number of ranks. Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs, but mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].		For example, in the case of [[:Category:Electronic minimization\|electronic minimization]], aim to set the number of ranks to the default value of {{TAG\|NBANDS}} divided by a small integer. Note that VASP will increase {{TAG\|NBANDS}} to accomodate the number of ranks. Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs, but mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].
	The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]].		The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]].
	Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points.		Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points.

Huebsch at 10:00, 12 April 2022

2022-04-12T10:00:32Z

← Older revision		Revision as of 10:00, 12 April 2022
Line 1:		Line 1:
	~~The best~~ [[:Category:Parallelization\|parallelization]] setup of a VASP calculation ~~needs~~ to ~~be tested~~ for each system, algorithm and computer architecture. Below, we offer general advice on how to optimize the parallelization.		To find the optimal [[:Category:Parallelization\|parallelization]] setup of a VASP calculation, it is necessary to run tests for each system, algorithm and computer architecture. Below, we offer general advice on how to optimize the parallelization.

	__TOC__		__TOC__

	==Optimizing the parallelization==		==Optimizing the parallelization==

			Basically, a few steps of any repetitive tasks give a good estimate of the performance for the full calculation.
			For example, run only a few electronic or ionic self-consistency steps (without reaching full convergence) and compare various setups for the parallelization.

	Try to get as close as possible to the actual system. This includes both the physical system (atoms, cell size, cutoff, ...) as well as the computational hardware (CPUs, interconnect, number of nodes, ...).		Try to get as close as possible to the actual system. This includes both the physical system (atoms, cell size, cutoff, ...) as well as the computational hardware (CPUs, interconnect, number of nodes, ...).
	If too many parameters are different, the parallel configuration may not be transferable to the production calculation.		If too many parameters are different, the parallel configuration may not be transferable to the production calculation.

	~~A few steps of any repetitive tasks give a good estimate of~~ the ~~performance~~ for the ~~full calculation~~.		It can be useful to first run VASP with {{TAG\|ALGO}}=''None'' set in the {{FILE\|INCAR}} file. That is because the [[:Category:Calculation setup\|computational setup]] for the [[:Category:Electronic minimization\|electronic minimization]] is done without actually performing the minimization. For instance, the FFTs are planned, and the irreducible '''k''' points of the first Brillouin zone are constructed. Therefore, some parameters, e.g., the default number of Kohn-Sham orbitals ({{TAG\|NBANDS}}) and the total number of plane waves, are written to the {{FILE\|OUTCAR}} file while using barely any computational time.
	For ~~example~~, ~~run only a few electronic or ionic self~~-~~consistency steps~~ (~~without reaching full convergence~~) and ~~compare various setups for~~ the ~~parallelization~~.

	Often, VASP yields the best performance by combining multiple parallelization options. This is because the parallel efficiency of each level drops near its limit.		Often, VASP yields the best performance by combining multiple parallelization options. This is because the parallel efficiency of each level drops near its limit.
	~~For~~ the ~~default option~~ (~~band~~ parallelization), the ~~limit is~~ {{TAG\|NBANDS}} divided by a small integer.		By default, VASP distributes the number of bands ({{TAG\|NBANDS}}) over the available MPI ranks. But it is often beneficial to add parallelization of the FFTs ({{TAG\|NCORE}}), parallelization over '''k''' points ({{TAG\|KPAR}}), and parallelization over separate calculations ({{TAG\|IMAGES}}). Additionally, there are some parallelization options for specific algorithms in VASP, e.g., {{TAG\|NOMEGAPAR}} for parallelization over imaginary frequency points in <math>GW</math> and RPA calculations. In summary, VASP parallelizes with
	Note that VASP will increase {{TAG\|NBANDS}} to match the number of ranks.
	Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs.		::<math>
	~~Recall that OpenMP and OpenACC enforce~~ that {{TAG\|NCORE}} ~~is not~~ set.		\text{total ranks} = \text{ranks parallelizing bands} \times \text{NCORE} \times \text{KPAR} \times \text{IMAGES} \times \text{other algorithm-dependent tags}.
	The '''k'''-point parallelization is efficient but requires additional memory.		</math>
	Given sufficient memory, increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points.
			First, create a list of the relevant [[Category:Parallelization\|INCAR tags for parallelization]] for the specific calculation. Then, the documentation for each of the relevant [[Category:Parallelization\|INCAR tags for parallelization]] should help to find reasonable settings to run test calculations.

			For example, in the case of [[:Category:Electronic minimization\|electronic minimization]], aim to set the number of ranks to the default value of {{TAG\|NBANDS}} divided by a small integer. Note that VASP will increase {{TAG\|NBANDS}} to match the number of ranks. Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs, but mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].
			The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]].
			Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points.
	Keep in mind that {{TAG\|KPAR}} should factorize the number of '''k''' points.		Keep in mind that {{TAG\|KPAR}} should factorize the number of '''k''' points.
	Finally, use the {{TAG\|IMAGES}} tag to split several VASP runs into separate calculations.		Finally, use the {{TAG\|IMAGES}} tag to split several VASP runs into separate calculations.

Huebsch: Created page with "The best parallelization setup of a VASP calculation needs to be tested for each system, algorithm and computer architecture. Below, we offer gen..."

2022-04-11T13:52:34Z

Created page with "The best parallelization setup of a VASP calculation needs to be tested for each system, algorithm and computer architecture. Below, we offer gen..."

New page

The best [[:Category:Parallelization|parallelization]] setup of a VASP calculation needs to be tested for each system, algorithm and computer architecture. Below, we offer general advice on how to optimize the parallelization.

__TOC__

==Optimizing the parallelization==

Try to get as close as possible to the actual system. This includes both the physical system (atoms, cell size, cutoff, ...) as well as the computational hardware (CPUs, interconnect, number of nodes, ...).
If too many parameters are different, the parallel configuration may not be transferable to the production calculation.

A few steps of any repetitive tasks give a good estimate of the performance for the full calculation.
For example, run only a few electronic or ionic self-consistency steps (without reaching full convergence) and compare various setups for the parallelization.

Often, VASP yields the best performance by combining multiple parallelization options. This is because the parallel efficiency of each level drops near its limit.
For the default option (band parallelization), the limit is {{TAG|NBANDS}} divided by a small integer.
Note that VASP will increase {{TAG|NBANDS}} to match the number of ranks.
Choose {{TAG|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs.
Recall that OpenMP and OpenACC enforce that {{TAG|NCORE}} is not set.
The '''k'''-point parallelization is efficient but requires additional memory.
Given sufficient memory, increase {{TAG|KPAR}} up to the number of irreducible '''k''' points.
Keep in mind that {{TAG|KPAR}} should factorize the number of '''k''' points.
Finally, use the {{TAG|IMAGES}} tag to split several VASP runs into separate calculations.
The limit is dictated by the number of desired calculations.

==Related tags an articles==
[[:Category:Parallelization|Parallelization]] , {{TAG|KPAR}}, {{TAG| NCORE}}, {{TAG|KPAR}}, {{TAG|IMAGES}}

==References==

[[Category:Parallelization]][[Category:Howto]]

← Older revision		Revision as of 10:10, 28 February 2025
Line 34:		Line 34:
	* Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs. Mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].		* Choose {{TAG\|NCORE}} as a factor of the cores per node to avoid communicating between nodes for the FFTs. Mind that {{TAG\|NCORE}} cannot be set with [[Combining MPI and OpenMP\|OpenMP]] threading and/or the [[OpenACC GPU port of VASP\|OpenACC GPU port]].
	* The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]]. Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points. Keep in mind that {{TAG\|KPAR}} should factorize the number of '''k''' points.		* The '''k'''-point parallelization is efficient but requires additional [[:Category:Memory\|memory]]. Given sufficient [[:Category:Memory\|memory]], increase {{TAG\|KPAR}} up to the number of irreducible '''k''' points. Keep in mind that {{TAG\|KPAR}} should factorize the number of '''k''' points.
			* For bulk systems with small unit cells ({{TAG\|NBANDS}} is small, NKPTS=no of k points is large), {{TAGO\|NCORE\|1}} and {{TAGO\|KPAR\|NKPTS}} is optimal.
			* For running on GPUs, you may try
			** MPI ranks = no of GPUs
			** {{TAGO\|KPAR\|no of GPUs}} if memory allows
			** [[Combining_MPI_and_OpenMP\|OpenMP threading]] can be quite important for the parts that still run on the host, because usually no of GPUs is rather small.
	* Finally, use the {{TAG\|IMAGES}} tag to split several VASP runs into separate calculations. The limit is dictated by the number of desired calculations.		* Finally, use the {{TAG\|IMAGES}} tag to split several VASP runs into separate calculations. The limit is dictated by the number of desired calculations.

← Older revision		Revision as of 09:59, 20 October 2023
Line 37:		Line 37:

	==Related tags an articles==		==Related tags an articles==
	[[:Category:Parallelization\|Parallelization]] , {{TAG\|KPAR}}, {{TAG\| NCORE}}, {{TAG\|KPAR}}, {{TAG\|IMAGES}}		[[:Category:Parallelization\|Parallelization]], {{TAG\|KPAR}}, {{TAG\| NCORE}}, {{TAG\|KPAR}}, {{TAG\|IMAGES}}

	==References==		==References==

	[[Category:Parallelization]][[Category:Howto]]		[[Category:Parallelization]][[Category:Howto]]