[SIESTA Logo]

SIESTA School 2024: MareNostrum 5

MareNostrum 5 is the largest operational supercomputer of the Barcelona Supercomputing Center. Each node contains 2 Intel Xeon Platinum processors (Sapphire Rapids, with 56 cores each) and 256 GB of RAM.

You will receive your BSC credentials in your institutional email account on Monday 11th November, in the morning. Your BSC account is already enabled. Please contact the organisers if you do not receive this email.

Connecting to MN5

From your computer terminal, use your SSH client to connect to a MN5 login node. There are two login nodes, glogin1.bsc.es, glogin2.bsc.es. You can connect to any of them, they are used interchangeably in the examples below.

Via conventional SSH client (Mac, Linux, Windows with WSL/Cygwin)

From a terminal, run

$ ssh <USERNAME>@glogin1.bsc.es

where <USERNAME> is your BSC username (e.g., ctn99999). You will be prompted for the corresponding password.

Via PuTTY or mobaXterm

If you don’t have any SSH client installed in your computer, please download and install mobaXterm (recommended) or PuTTY.

When you open the graphical interface define the server/host name (glogin1.bsc.es) and provide your username and password when prompted. For mobaXterm, there is a short generic demo video available here.

For visualizations using PuTTY, please keep in mind the following:

In mobaXterm this is enabled by default.

Changing your password

Please change your password as soon as possible. In order to do this, connect via ssh to transfer1.bsc.es and execute the passwd command; the prompt will ask you for your existing password (once) and for your new password (twice). The new password takes about 10 minutes to be effective. A sample session for changing the password is:

$ ssh <USERNAME>@transfer1.bsc.es

[<USERNAME>@transfer1 ~]$ passwd
Changing password for username.
Old Password:
New Password:
Reenter New Password:
Password changed.

Note that you need to connect to transfer1.bsc.es directly from outside of BSC: if you are connected to, say, glogin1.bsc.es, you will not be able to successfully SSH from there to transfer1.bsc.es.

Submitting a job

Calculations in MareNostrum 5 need to be submitted through the queing system. For that purpose, we will be using a job script, provided in /gpfs/projects/nct_315/SCRIPTS/runmn.sh. You will need to copy this script to each tutorial folder in which you intend to run siesta.

The contents of the script are:

#!/bin/bash
#SBATCH -J tutorialXX
#SBATCH -n 4
#SBATCH -t 0:30:00
#SBATCH -o %x-%j.out
#SBATCH -e %x-%j.err
#SBATCH -q gp_training
#SBATCH -A nct_315
#SBATCH -D .
##
## For reservations, uncomment the following line.
## Change X by the current day of the school (1-5)
##SBATCH --reservation=SIESTA24-DAY1X

# DO NOT CHANGE THIS LINE
source /gpfs/projects/nct_315/SIESTA/siestarc.sh

# EDIT TO ADD THE CORRECT INPUT AND OUTPUT FILES.
srun -n 4 siesta < input.fdf > output.out

Note that input.fdf and output.out are generic input and output file names, for each execution of siesta you will need to change them to the actual names of your input and output files. It is also convenient to change the name of the job (#SBATCH -J option) to something that allows you to identify (vs. your other submissions) when checking the status of the queue.

The -n options in #SBATCH and srun indicate the number of computing cores used. Please do not needlessly increase these quantities, be mindful that resources are limited and must be shared between all of the students.

You can submit this job script from a login node by executing:

$ sbatch runmn.sh

For more information on running jobs, visit and read the MareNostrum guides.

SLURM reservations

BSC has enabled daily reservations for us. They will be available 12:00-19:00, every day of the school:

These reservations will consist of 6 MN5 nodes, i.e., roughly 4 CPU cores per participant in the School.

Outside from these hours, there will be no specific reservation for the School, but your account will still have access to the training queue.

Directory with tutorials

For every practical there is a folder in the shared directory /gpfs/projects/nct_315/TUTORIALS that contains all the files you will need for said practical. At the start of every tutorial, please copy the required files from that shared directory to your work directory. For example, for the practical of the first day,

$ cp -pr /gpfs/projects/nct_315/TUTORIALS/day1 .

Please do NOT copy these files before the start of the practical, in case they are updated shortly before the practical starts.

Visualization

Images should be viewed on your own PCs. Visualizations done directly on the MareNostrum login nodes will result in account banning.

In order to copy the files to your local computer (laptop/desktop), see below.

Tools used for visualization

We recommend that you install in your local computer the following software, which will be used during the school:

Gnuplot is available as a package on most linux distributions. Ovito probably will need to be downloaded from their website.

Other software you might be interested in includes:

Accessing the node reservations.

In this case you will need to connect directly (interactively) to a compute node instead of submitting a job script. In order to do this, you need to execute the following command from one of the login nodes of MareNostrum 5:

$ salloc -t 4:00:00 -n 1 -c 1 --reservation=SIESTA24-DAY1X -J siestaschool

The X in -DAY1X needs to be changed to 1, 2, 3, 4 or 5, according to the day of the school.

This will create a 4-hour long reservation (-t option), with 1 task (-n) that uses 1 CPU core (-c). The -J option is again an identifier that you can change at will.

Again, please do not needlessly increase these quantities, be mindful that resources are limited and must be shared between all of the students.

Once inside this interactive session, make sure that you load the Siesta school environment before doing anything:

$ source /gpfs/projects/nct_315/SIESTA/siestarc.sh

This will give you access to the siesta binary and to the rest of the tools of the school.

Beware that the Siesta school environment will stay active as long as your interactive session is active. If you logout or otherwise lose conntection to the compute node, you will need to load the environment again when you reconnect.

We encourage you to run visualizations locally on your own computer.

Copying and backing up files

You can backup your files in case you want to keep them after the school is finished. You need to do this from your local machine (you cannot start outgoing connections from BSC machines). It is recommended that you transfer your files from one of the data transfer nodes (transfer1.bsc.es) rather than from one of the login nodes.

You can transfer files in multiple ways:

$ scp -pr <USERNAME>@transfer1.bsc.es:/path/to/files .

Data preservation

You will lose access to your account on November 15th or shortly afterwards. We strongly recommend that you perform regular (at least daily) backups of your BSC files to prevent the loss of any data.

Other