{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Using Climada on the Euler Cluster (ETH internal)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Content\n", "\n", "- [Access to Euler](#Access-to-Euler)\n", "- [Installation and working directories](#Installation-and-working-directories)\n", "- [Pre-installed version of Climada](#Pre-installed-version-of-Climada)\n", "- [Working with Git branches](#Working-with-Git-branches)\n", "- [Fallback: Conda installation](#Fallback:-Conda)\n", "- [Run a Jupyter Notebook on Euler](#Run-Jupyter-Notebook-on-Euler)\n", "- [Trouble shooting](#Trouble-shooting)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Access to Euler\n", "\n", "See https://scicomp.ethz.ch/wiki/Getting_started_with_clusters for details on how to register at and get started with Euler.\n", "\n", "For all steps below, first enter the Cluster via SSH." ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "\n", "## Installation and working directories\n", "\n", "Please, get familiar with the various Euler storage options: https://scicomp.ethz.ch/wiki/Storage_systems.
\n", "As a general rule: use `/cluster/project` for installation and `/cluster/work` for data processing.\n", "\n", "For ETH WCR group members, the suggested installation and working directories are `/cluster/project/climate/$USER` and `/cluster/work/climate/$USER` respectively.
\n", "You may have to create the installation directory:\n", "\n", "```bash\n", "mkdir -p /cluster/project/climate/$USER \\\n", " /cluster/work/climate/$USER\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Pre-installed version of Climada\n", "\n", "Climada is pre-installed and available in the default pip environment of Euler." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 1. Load dependencies\n", "\n", "```bash\n", "env2lmod\n", "module load gcc/8.2.0 python/3.9.9 gdal/3.4.3 geos/3.9.1 proj/8.2.1 libspatialindex/1.9.3 netcdf/4.6.3 eccodes/2.22.1 zlib/1.2.9 libtiff/4.2.0 sqlite/3.35.5\n", "```\n", "\n", "You need to execute these two lines every time you login to Euler before Climada can be used. To safe yourself from doing it manually, one can append these lines to the ~/.bashrc script, which is automatically executed upon logging in to Euler." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 2. Check installation \n", "\n", "```bash\n", "python -c 'import climada; print(climada.__file__)'\n", "```\n", " should output something like this:\n", "```bash\n", "/cluster/apps/nss/gcc-8.2.0/python/3.9.9/x86_64/lib64/python3.9/site-packages/climada/__init__.py\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 3. Adjust the Climada configuration\n", "\n", "Edit a configuration file according to your needs (see [Guide_Configuration](../guide/Guide_Configuration.ipynb)).\n", "Create a climada.conf file e.g., in /cluster/home/$USER/.config with the following content:\n", "\n", "```json\n", "{\n", " \"local_data\": {\n", " \"system\": \"/cluster/work/climate/USERNAME/climada/data\",\n", " \"demo\": \"/cluster/project/climate/USERNAME/climada_python/data/demo\",\n", " \"save_dir\": \"/cluster/work/climate/USERNAME/climada/results\"\n", " }\n", "}\n", "```\n", "(Replace USERNAME with your nethz-id.)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 4. Run a job\n", "\n", "Please see the docs at https://slurm.schedmd.com/ on how to use the `slurm` batch system \n", "and the Wiki https://scicomp.ethz.ch/wiki/Transition_from_LSF_to_Slurm for a mapping of `lsf` commands to their `slurm` equivalents.\n", "\n", "```bash\n", "cd /cluster/work/climate/$USER # change to the working directory\n", "sbatch [slurm options*] --wrap 'python climada_job_script.py' # submit the job\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Working with Git branches\n", "\n", "If the Climada version of the default installation is not according to your needs, you can install Climada from a local Git repository." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 1. Load dependencies \n", "\n", "See [Load dependencies](#1.-Load-dependencies) above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 2. Create installation environment\n", "\n", "```bash\n", "python -m venv --system-site-packages /cluster/project/climate/$USER/climada_venv\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 3. Checkout sources\n", "\n", "```bash\n", "cd /cluster/project/climate/$USER\n", "git clone https://github.com/CLIMADA-project/climada_python.git\n", "cd climada_python\n", "git checkout develop # i.e., your branch of interest\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 4. Pip install Climada\n", "\n", "```bash\n", "source /cluster/project/climate/$USER/climada_venv/bin/activate\n", "pip install -e /cluster/project/climate/$USER/climada_python\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 5. Check installation\n", "\n", "```bash\n", "cd /cluster/work/climate/$USER\n", "python -c 'import climada; print(climada.__file__)'\n", "```\n", "\n", "should output exactly this (with explicit \\$USER):\n", "```bash\n", "/cluster/project/climate/$USER/climada_python/climada/__init__.py\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 6. Adjust the Climada configuration\n", "\n", "See [Adjust the Climada configuration](#3.-Adjust-the-Climada-configuration) above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 7. Run a job\n", "\n", "See [Run a job](#4.-Run-a-job) above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Fallback: Conda\n", "\n", "If Climada cannot be installed through pip because of changed dependency requirements, there is still the possibility to install Climada through the Conda environment.\n", "> **WARNING:** This approach is highly discouraged, as it imposes a heavy and mostly unnecessary burden on the file system of the cluster." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Installation\n", "\n", "#### 1. Conda\n", "\n", "Download or update to the latest version of [Miniconda](https://conda.io/miniconda.html).
\n", "Installation is done by execution of the following steps:\n", "\n", "```bash\n", "cd /cluster/project/climate/USERNAME\n", "wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n", "bash Miniconda3-latest-Linux-x86_64.sh\n", "miniconda3/bin/conda init\n", "rm Miniconda3-latest-Linux-x86_64.sh\n", "```\n", "\n", "During the installation process of Miniconda, you are prompted to set the working directory according to your choice.\n", "Set it to `/cluster/project/climate/USERNAME/miniconda3`.
\n", "Once the installation has finished, log out of Euler and in again. The command prompt should be preceded by `(base)`, \n", "indicating that the installation was a success and that you login in into conda's base environment by default." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 2. Checkout sources \n", "\n", "See [Checkout sources](#3.-Checkout-sources) above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3. Climada environment\n", "\n", "Create the conda environment:\n", "\n", "```bash\n", "cd /cluster/project/climate/USERNAME/climada_python\n", "conda env create -f requirements/env_climada.yml --name climada_env\n", "conda env update -n climada_env -f requirements/env_developer.yml\n", "\n", "conda activate climada_env\n", "conda install conda-build\n", "conda develop .\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 4. Adjust the Climada configuration\n", "\n", "See [Adjust the Climada configuration](#3.-Adjust-the-Climada-configuration) above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 5. Climada scripts \n", "\n", "Create a bash script for executing python scripts in the climada environment, `climadajob.sh`:\n", "```bash\n", "#!/bin/bash\n", "PYTHON_SCRIPT=$1\n", "shift\n", ". ~/.bashrc\n", "conda activate climada_env\n", "python $PYTHON_SCRIPT $@\n", "echo $PYTHON_SCRIPT completed\n", "```\n", "\n", "Make it executable:\n", "\n", "```bash\n", "chmod +x climadajob.sh\n", "```\n", "\n", "Create a python script that executes climada code, e.g., `climada_smoke_test.py`:\n", "\n", "```python\n", "import sys\n", "from climada import CONFIG, SYSTEM_DIR\n", "from climada.util.test.test_finance import TestNetpresValue\n", "TestNetpresValue().test_net_pres_val_pass()\n", "print(SYSTEM_DIR)\n", "print(CONFIG.local_data.save_dir.str())\n", "print(\"the script ran with arguments\", sys.argv)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 6. Run a job \n", "\n", "With the scripts from above you can submit the python script as a job like this:\n", "\n", "```bash\n", "sbatch [slurm options] --wrap \"/path/to/climadajob.sh /path/to/climada_smoke_test.py arg1 arg2\"\n", "```\n", "\n", "After the job has finished the slurm output file should look something like this:\n", "\n", "```\n", "/cluster/work/climate/USERNAME/climada/data\n", "/cluster/work/climate/USERNAME/climada/results\n", "the script ran with arguments ['/path/to/climada_smoke_test.py', 'arg1' 'arg2']\n", "python_script.sh completed\n", "```\n", "\n", "Please see the docs at https://slurm.schedmd.com/ on how to use the `slurm` batch system \n", "and the Wiki https://scicomp.ethz.ch/wiki/Transition_from_LSF_to_Slurm for a mapping of `lsf` commands to their `slurm` equivalents." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Deinstallation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 1. Conda \n", "\n", "Remove the miniconda3 directory from the installation directory:\n", "\n", "```bash\n", "rm -rf /cluster/project/climate/USERNAME/miniconda3/\n", "```\n", "\n", "Delete the conda related parts from `/cluster/home/USERNAME/.bashrc`, i.e., everything between\n", "\n", "`# >>> conda initialize >>>`\\\n", "and\\\n", "`# <<< conda initialize <<<`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 2. Climada\n", "\n", "Remove the climada sources and config file:\n", "\n", "```bash\n", "rm -rf /cluster/project/climate/USERNAME/climada_python\n", "rm -f /cluster/home/USERNAME/climada.conf /cluster/home/USERNAME/*/climada.conf\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Run Jupyter Notebook on Euler" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is possible to run a Jupyter Notebook on Euler and access it through a browser from your local machine.\n", "To make this possible you have to do some preparation steps on both, your local machine and on Euler.\n", "\n", "**Windows Users:** to run the commands given below on your local computer you have to use `git-bash` (or WSL `bash` if Ubuntu on Windows is installed)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 1. Get the _Jupyter on Euler_ scripts and documentation\n", "on your local computer, run these commands:\n", "\n", "```bash\n", "git clone git@gitlab.ethz.ch:sfux/Jupyter-on-Euler-or-Leonhard-Open.git\n", "cd Jupyter-on-Euler-or-Leonhard-Open\n", "git checkout add_slurm\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 2. Create an SSH key pair\n", "\n", "on your local computer, run this command:\n", "\n", "```bash\n", "ssh-keygen ssh-keygen -t ed25519 -f $HOME/.ssh/id_ed25519_euler\n", "```\n", "\n", "When prompted, enter a _strong_ passphrase.\n", "\n", "Then, run\n", "\n", "```bash\n", "ssh-add $HOME/.ssh/id_ed25519_euler\n", "```\n", "\n", "and provide the passphrase from above.\n", "\n", "**Windows Users:** `ssh-add` is working together with an SSH-Agent, which is not running out of the box in `git-bash`. Follow the instructions [here](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/working-with-ssh-key-passphrases) to make it available." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 3. Copy the public key into the authorized_keys file on Euler\n", "\n", "```bash\n", "ssh-copy-id -i $HOME/.ssh/id_ed25519_euler.pub USERNAME@euler.ethz.ch\n", "```\n", "\n", "replace `USERNAME` with your ETH user name. When prompted to enter the passwort provide your LADP password (i.g., it is the same as your E-mail password).\n", "\n", "At this point you should be able to access Euler through `ssh` without password:\n", "\n", "```bash\n", "ssh USERNAME@euler.ethz.ch\n", "```\n", "should bring you directly to the Euler cluster." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 4. Prepare Euler environment\n", "\n", "It is highly recommanded to use a python environment and a github clone as described above under **Working with Git Branches**.\n", "\n", "Login to Euler.\n", "\n", "Unless you have done so already you need to permanently switch to the new software stack:\n", "```bash\n", "set_software_stack.sh new\n", "```\n", "\n", "Unless you have done so already you need to append the default module selection in your `~/.bashrc` file:\n", "```bash\n", "echo >> ~/.bashrc\n", "echo module load gcc/8.2.0 python/3.9.9 gdal/3.4.3 geos/3.9.1 proj/8.2.1 libspatialindex/1.9.3 netcdf/4.6.3 eccodes/2.22.1 zlib/1.2.9 libtiff/4.2.0 sqlite/3.35.5 >> ~/.bashrc\n", "```\n", "\n", "Activate your python environment:\n", "```bash\n", "source /cluster/project/climate/$USER/climada_venv/bin/activate\n", "```\n", "\n", "Install an IPyhton-kernel:\n", "```\n", "python -m ipykernel install --user --name climada_venv\n", "``` \n", "\n", "Logout." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 5. Adjust the script\n", "\n", "Open the script `start_jupyter_nb.sh` in an editor and change the gcc and python version numbers in lines 421/422 so they look like this:\n", "```\n", "[421] JNB_MODULE_COMMAND=\"gcc/8.2.0 r/4.0.2 python/3.9.9 eth_proxy $JNB_JULIA\"\n", "[422] echo -e \"Using new software stack (gcc/8.2.0 python/3.9.9 eth_proxy $JNB_JULIA)\"\n", "\n", "```\n", "\n", "make the script executable:\n", "```bash\n", "chmod +x start_jupyter_nb.sh\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### 6. Start a Jupyter Notebook on Euler\n", "\n", "At this point everything is ready. You should be able to start a Jupter Notebook remotely running on Euler.\n", "\n", "Check out the script:\n", "```bash\n", "./start_jupyter_nb.sh\n", "```\n", "\n", "Run Jupyter on Euler on a single cpu (default) with 1G of memory (default) in your projects directory with the slurm batch system:\n", "```bash\n", "./start_jupyter_nb.sh -u NETHZID -b SLURM -w /cluster/project/climate/USERNAME\n", "```\n", "\n", "If all went fine your default browser will be opened after some time with Jupyter running in your project directory.\\\n", "When opening or creating a notebook make sure you select the `climada_venv` kernel." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Trouble shooting\n", "\n", "### 1. Python Module not found or available\n", "- Make sure your python environment is activated.\n", "- Run `pip install --upgrade MISSING_MODULE`." ] } ], "metadata": { "kernelspec": { "display_name": "climada_env", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:05:47) \n[Clang 12.0.1 ]" }, "vscode": { "interpreter": { "hash": "4aebf7f26d9a9d4c9696d8ddcd034589cd11abb7fe515057c687f2f3cec840ea" } } }, "nbformat": 4, "nbformat_minor": 4 }