It’s been close to a year since I updated our cluster; I was going to do it over Christmas, but never quite got around to it. The period of social distancing (and procrastinating on my research) is a great time, right? The cluster is running Centos 7. The biggest issue with upgrading it is the…
Author: kfrye
Lustre Woes
I’m currently maintaining a small cluster with a head node and 17 other nodes with everything connected together through a switch. It’s running Centos 7 and has a Lustre file sytem which is contained on three additional machines; one of them is running the MGS/MDS servers, and the other two are running two storage targets…
A tale of linker woes
I wanted to compile MADbench2, which is a program designed to test the interaction of I/O with communication in an HPC environment. It has some prerequisites such as Scalapack, Lapack and their prerequisites. I have root access on this particular cluster, so I was hoping I could just install a few precompiled packages and just…
Building OpenMPI
First step is to download the latest stable version from here: https://www.open-mpi.org/software/ompi/v4.0/ After opening the tarball, go into the directory and configure: ./configure –prefix=/opt/openmpi-4.X.X –enable-mpi1-compatibility Substitute X.X for the actual version being installed. This will allow you to have multiple versions of openmpi on a system, which may be needed for existing applications or research…
Git public keys
A server that I frequently use (and don’t have root access to) doesn’t start its ssh-agent automatically nor register the existing public keys. To start the ssh agent: eval “$(ssh-agent -s)” To add your key to the ssh agent: ssh-add ~/.ssh/id_rsa
More on PMPI
Many scientific computing programs, including those using MPI, are coded in Fortran. PMPI wrappers coded in C can be used with these Fortran codes, but it’s a bit complicated because Fortran is case-sensitive and sometimes function names contain underscores not present in the C-equivalent. So a program might use mpi_send or __mpi_send, neither of which…
PMPI
MPI comes with a built-in interface for performing measurements and other actions during MPI calls. Here’s a quick tutorial on how it works: Create a file called wrap.c: #include <mpi.h>#include <stdio.h>double start, end;int ret;int MPI_Send(const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm) { start = MPI_Wtime(); ret = PMPI_Send(buf,…
Installations with Spack
As a user of large-scale Linux systems both at a university and at a national laboratory, I often find myself needing to use software that isn’t available at a system level. In some cases, it’s possible to make a request for that application to be added by the system administrators, but, more often than not,…