ScaLapack on the Grendel cluster

ScaLapack is a linear algebra library, which runs across nodes in a cluster environment. It depends on several libraries:

MPI The Message Passing Interface provides the (primitive) basic communication layer.
There exists several implementations of MPI (and the newer MPI2). Some of the most popular are:
MPICH1, MPICH2, LAM-MPI, OpenMPI.
BLACS The Basic Linear Algebra Communication Subprograms sitting on top of MPI.
BLAS The Basic Linear Algebra Subroutines. These subroutines performs vector-vector, matrix-vector
and matrix-matrix operations. Computer vendors often provides these libraries optimized for their hardware and highly efficient.
They may be multithreaded (within one node) but normally they cannot utillize more nodes. On Grendel are installed both
Intel's MKL library and AMD's ACML library.
Lapack The Linear Algebra PACKage , is a collection of routines for solving linear systems, least squares problems, eigenproblems, and singular problems. Lapack uses the BLAS-routines, and is included in both the MKL- and ACML- libraries .
PBLAS The Parallel Basic Linear Algebra Subroutines. This library is sitting on top of BLACS (MPI) and BLAS.
It provides parallel BLAS routines which runs across nodes.
ScaLapack The Scalable LAPACK library contains a subset of the wellknown Lapack routines, designed for distributed memory parallel computers (clusters). It depends on all libraries above!

On Grendel ScaLapack is installed in two versions, the only difference is the MPI-library used.
In /com/lib/scalapack_mpich1 the libraries are build with Intel's compiler, ACML BLAS, MPICH1, BLACS 1.1p3, ScaLapack 1.8.0.
In /com/lib/scalapack_openmpi the libraries are build with Intel's compiler, ACML BLAS, OpenMPI, BLACS 1.1p3, ScaLapack 1.8.0.
The main difference is that OpenMPI is a more modern implementation of MPI, able to communicate
via the queueingsystem infrastructure, obsoleting the needs for hostfiles etc.
In both directories mentioned above there is a file called SLmake.inc which is to be included in Makefiles to provide the macros needed
to compile and link programs using ScaLapack.
SLmake.inc also contains informations how to run the program, especially how to define a proper
LD_LIBRARY_PATH environment variable needed to point out all the (dynamically loaded) libraries used by the program.
See also How to use ScaLapack on Grendel on our faq page for detailed informations how to compile and run ScaLapack-programs.

Back


		ABOUT-DCSC	DCSC/SDU	DCSC/AU	DCSC/AAU	DCSC/DTU	DCSC/KU

	+Open all -Close all About DCSC /AU Home Information About DCSC /AU Organisation News Current News News Archive Resources Grendel cluster Grendel GPU cluster Huge IBM p6-cluster Freke SGI Altix Sleipner IBM p4-cluster The Archive Grid Computing Statistics Utilization Graphs System Load Support How to Get Help How to Get a Grant How to Get an Account Utillities Links Contact DCSC /AU Contact DCSC Central Office Other DCSC Centers
			Overview Hardware Software Batchjobs Hints ScaLapack on the Grendel cluster ScaLapack is a linear algebra library, which runs across nodes in a cluster environment. It depends on several libraries: MPI The Message Passing Interface provides the (primitive) basic communication layer. There exists several implementations of MPI (and the newer MPI2). Some of the most popular are: MPICH1, MPICH2, LAM-MPI, OpenMPI. BLACS The Basic Linear Algebra Communication Subprograms sitting on top of MPI. BLAS The Basic Linear Algebra Subroutines. These subroutines performs vector-vector, matrix-vector and matrix-matrix operations. Computer vendors often provides these libraries optimized for their hardware and highly efficient. They may be multithreaded (within one node) but normally they cannot utillize more nodes. On Grendel are installed both Intel's MKL library and AMD's ACML library. Lapack The Linear Algebra PACKage , is a collection of routines for solving linear systems, least squares problems, eigenproblems, and singular problems. Lapack uses the BLAS-routines, and is included in both the MKL- and ACML- libraries . PBLAS The Parallel Basic Linear Algebra Subroutines. This library is sitting on top of BLACS (MPI) and BLAS. It provides parallel BLAS routines which runs across nodes. ScaLapack The Scalable LAPACK library contains a subset of the wellknown Lapack routines, designed for distributed memory parallel computers (clusters). It depends on all libraries above! On Grendel ScaLapack is installed in two versions, the only difference is the MPI-library used. In `/com/lib/scalapack_mpich1` the libraries are build with Intel's compiler, ACML BLAS, MPICH1, BLACS 1.1p3, ScaLapack 1.8.0. In `/com/lib/scalapack_openmpi` the libraries are build with Intel's compiler, ACML BLAS, OpenMPI, BLACS 1.1p3, ScaLapack 1.8.0. The main difference is that OpenMPI is a more modern implementation of MPI, able to communicate via the queueingsystem infrastructure, obsoleting the needs for hostfiles etc. In both directories mentioned above there is a file called `SLmake.inc` which is to be included in Makefiles to provide the macros needed to compile and link programs using ScaLapack. SLmake.inc also contains informations how to run the program, especially how to define a proper LD_LIBRARY_PATH environment variable needed to point out all the (dynamically loaded) libraries used by the program. See also How to use ScaLapack on Grendel on our faq page for detailed informations how to compile and run ScaLapack-programs. Back