Using Climada on the Euler Cluster (ETH internal)#

Content#

  • Access to Euler

  • Installation and working directories

  • Pre-installed version of Climada

  • Working with Git branches

  • Fallback: Conda installation

  • Run a Jupyter Notebook on Euler

  • Trouble shooting

Access to Euler#

See https://scicomp.ethz.ch/wiki/Getting_started_with_clusters for details on how to register at and get started with Euler.

For all steps below, first enter the Cluster via SSH.

Installation and working directories#

Please, get familiar with the various Euler storage options: https://scicomp.ethz.ch/wiki/Storage_systems.
As a general rule: use /cluster/project for installation and /cluster/work for data processing.

For ETH WCR group members, the suggested installation and working directories are /cluster/project/climate/$USER and /cluster/work/climate/$USER respectively.
You may have to create the installation directory:

mkdir -p /cluster/project/climate/$USER \
         /cluster/work/climate/$USER

Pre-installed version of Climada#

Climada is pre-installed and available in the default pip environment of Euler.

1. Load dependencies#

env2lmod
module load gcc/8.2.0 python/3.9.9 gdal/3.4.3 geos/3.9.1 proj/8.2.1 libspatialindex/1.9.3 netcdf/4.6.3 eccodes/2.22.1 zlib/1.2.9 libtiff/4.2.0 sqlite/3.35.5

You need to execute these two lines every time you login to Euler before Climada can be used. To safe yourself from doing it manually, one can append these lines to the ~/.bashrc script, which is automatically executed upon logging in to Euler.

2. Check installation#

python -c 'import climada; print(climada.__file__)'

should output something like this:

/cluster/apps/nss/gcc-8.2.0/python/3.9.9/x86_64/lib64/python3.9/site-packages/climada/__init__.py

3. Adjust the Climada configuration#

Edit a configuration file according to your needs (see Guide_Configuration). Create a climada.conf file e.g., in /cluster/home/$USER/.config with the following content:

{
    "local_data": {
        "system": "/cluster/work/climate/USERNAME/climada/data",
        "demo": "/cluster/project/climate/USERNAME/climada/data/demo",
        "save_dir": "/cluster/work/climate/USERNAME/climada/results"
    }
}

(Replace USERNAME with your nethz-id.)

4. Run a job#

Please see the docs at https://slurm.schedmd.com/ on how to use the slurm batch system and the Wiki https://scicomp.ethz.ch/wiki/Transition_from_LSF_to_Slurm for a mapping of lsf commands to their slurm equivalents.

cd /cluster/work/climate/$USER  # change to the working directory
sbatch [slurm options*] --wrap 'python climada_job_script.py'  # submit the job

Working with Git branches#

If the Climada version of the default installation is not according to your needs, you can install Climada from a local Git repository.

1. Load dependencies#

See Load dependencies above.

2. Create installation environment#

python -m venv --system-site-packages /cluster/project/climate/$USER/climada_venv

3. Checkout sources#

cd /cluster/project/climate/$USER
git clone https://github.com/CLIMADA-project/climada_python.git
cd climada_python
git checkout develop  # i.e., your branch of interest

4. Pip install Climada#

source /cluster/project/climate/$USER/climada_venv/bin/activate
pip install -e /cluster/project/climate/$USER/climada_python

5. Check installation#

cd /cluster/work/climate/$USER
python -c 'import climada; print(climada.__file__)'

should output exactly this (with explicit $USER):

/cluster/project/climate/$USER/climada_python/climada/__init__.py

6. Adjust the Climada configuration#

See Adjust the Climada configuration above.

7. Run a job#

See Run a job above.

Fallback: Conda#

If Climada cannot be installed through pip because of changed dependency requirements, there is still the possibility to install Climada through the Conda environment.

WARNING: This approach is highly discouraged, as it imposes a heavy and mostly unnecessary burden on the file system of the cluster.

Installation#

1. Conda#

Download or update to the latest version of Miniconda.
Installation is done by execution of the following steps:

cd /cluster/project/climate/USERNAME
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
miniconda3/bin/conda init
rm Miniconda3-latest-Linux-x86_64.sh

During the installation process of Miniconda, you are prompted to set the working directory according to your choice. Set it to /cluster/project/climate/USERNAME/miniconda3.
Once the installation has finished, log out of Euler and in again. The command prompt should be preceded by (base), indicating that the installation was a success and that you login in into conda’s base environment by default.

2. Checkout sources#

See Checkout sources above.

3. Climada environment#

Create the conda environment:

cd /cluster/project/climate/USERNAME/climada_python
conda env create -f requirements/env_climada.yml --name climada_env
conda env update -n climada_env -f requirements/env_developer.yml

conda activate climada_env
conda install conda-build
conda develop .

4. Adjust the Climada configuration#

See Adjust the Climada configuration above.

5. Climada scripts#

Create a bash script for executing python scripts in the climada environment, climadajob.sh:

#!/bin/bash
PYTHON_SCRIPT=$1
shift
. ~/.bashrc
conda activate climada_env
python $PYTHON_SCRIPT $@
echo $PYTHON_SCRIPT completed

Make it executable:

chmod +x climadajob.sh

Create a python script that executes climada code, e.g., climada_smoke_test.py:

import sys
from climada import CONFIG, SYSTEM_DIR
from climada.util.test.test_finance import TestNetpresValue
TestNetpresValue().test_net_pres_val_pass()
print(SYSTEM_DIR)
print(CONFIG.local_data.save_dir.str())
print("the script ran with arguments", sys.argv)

6. Run a job#

With the scripts from above you can submit the python script as a job like this:

sbatch [slurm options] --wrap "/path/to/climadajob.sh /path/to/climada_smoke_test.py arg1 arg2"

After the job has finished the slurm output file should look something like this:

/cluster/work/climate/USERNAME/climada/data
/cluster/work/climate/USERNAME/climada/results
the script ran with arguments ['/path/to/climada_smoke_test.py', 'arg1' 'arg2']
python_script.sh completed

Please see the docs at https://slurm.schedmd.com/ on how to use the slurm batch system and the Wiki https://scicomp.ethz.ch/wiki/Transition_from_LSF_to_Slurm for a mapping of lsf commands to their slurm equivalents.

Deinstallation#

1. Conda#

Remove the miniconda3 directory from the installation directory:

rm -rf /cluster/project/climate/USERNAME/miniconda3/

Delete the conda related parts from /cluster/home/USERNAME/.bashrc, i.e., everything between

# >>> conda initialize >>>
and
# <<< conda initialize <<<

2. Climada#

Remove the climada sources and config file:

rm -rf /cluster/project/climate/USERNAME/climada_python
rm -f /cluster/home/USERNAME/climada.conf /cluster/home/USERNAME/*/climada.conf

Run Jupyter Notebook on Euler#

It is possible to run a Jupyter Notebook on Euler and access it through a browser from your local machine. To make this possible you have to do some preparation steps on both, your local machine and on Euler.

Windows Users: to run the commands given below on your local computer you have to use git-bash (or WSL bash if Ubuntu on Windows is installed).

1. Get the Jupyter on Euler scripts and documentation#

on your local computer, run these commands:

git clone git@gitlab.ethz.ch:sfux/Jupyter-on-Euler-or-Leonhard-Open.git
cd Jupyter-on-Euler-or-Leonhard-Open
git checkout add_slurm

2. Create an SSH key pair#

on your local computer, run this command:

ssh-keygen ssh-keygen -t ed25519 -f $HOME/.ssh/id_ed25519_euler

When prompted, enter a strong passphrase.

Then, run

ssh-add $HOME/.ssh/id_ed25519_euler

and provide the passphrase from above.

Windows Users: ssh-add is working together with an SSH-Agent, which is not running out of the box in git-bash. Follow the instructions here to make it available.

3. Copy the public key into the authorized_keys file on Euler#

ssh-copy-id -i $HOME/.ssh/id_ed25519_euler.pub USERNAME@euler.ethz.ch

replace USERNAME with your ETH user name. When prompted to enter the passwort provide your LADP password (i.g., it is the same as your E-mail password).

At this point you should be able to access Euler through ssh without password:

ssh USERNAME@euler.ethz.ch

should bring you directly to the Euler cluster.

4. Prepare Euler environment#

It is highly recommanded to use a python environment and a github clone as described above under Working with Git Branches.

Login to Euler.

Unless you have done so already you need to permanently switch to the new software stack:

set_software_stack.sh new

Unless you have done so already you need to append the default module selection in your ~/.bashrc file:

echo >> ~/.bashrc
echo module load gcc/8.2.0 python/3.9.9 gdal/3.4.3 geos/3.9.1 proj/8.2.1 libspatialindex/1.9.3 netcdf/4.6.3 eccodes/2.22.1 zlib/1.2.9 libtiff/4.2.0 sqlite/3.35.5 >> ~/.bashrc

Activate your python environment:

source /cluster/project/climate/$USER/climada_venv/bin/activate

Install an IPyhton-kernel:

python -m ipykernel install --user --name climada_venv

Logout.

5. Adjust the script#

Open the script start_jupyter_nb.sh in an editor and change the gcc and python version numbers in lines 421/422 so they look like this:

[421]            JNB_MODULE_COMMAND="gcc/8.2.0 r/4.0.2 python/3.9.9 eth_proxy $JNB_JULIA"
[422]            echo -e "Using new software stack (gcc/8.2.0 python/3.9.9 eth_proxy $JNB_JULIA)"

make the script executable:

chmod +x start_jupyter_nb.sh

6. Start a Jupyter Notebook on Euler#

At this point everything is ready. You should be able to start a Jupter Notebook remotely running on Euler.

Check out the script:

./start_jupyter_nb.sh

Run Jupyter on Euler on a single cpu (default) with 1G of memory (default) in your projects directory with the slurm batch system:

./start_jupyter_nb.sh -u NETHZID -b SLURM -w /cluster/project/climate/USERNAME

If all went fine your default browser will be opened after some time with Jupyter running in your project directory.
When opening or creating a notebook make sure you select the climada_venv kernel.

Trouble shooting#

1. Python Module not found or available#

  • Make sure your python environment is activated.

  • Run pip install --upgrade MISSING_MODULE.