{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Miscellaneous CLIMADA conventions" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [], "toc": true }, "source": [ "

Table of Contents

\n", "
\n", " \n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Dependencies (python packages) " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Python is extremely powerful thanks to the large amount of available libraries, packages and modules. However, maintaining a code with a large number of such packages creates dependencies which is very care intensive. Indeed, each package developer can and does update and develop continuously. This means that certain code can become obsolete over time, stop working altogether, or become incompatible with other packages. Hence, it is crucial to keep the philosophie:\n", "\n", "*As many packages as needed, as few as possible.*\n", "\n", "Thus, when you are coding, follow these priorities:\n", "\n", "1. [Python standard library](https://docs.python.org/3/library/index.html)\n", "2. Funktions and methods already implemented in CLIMADA (do NOT introduce circulary imports though)\n", "3. [Packages already included in CLIMADA](https://github.com/CLIMADA-project/climada_python/network/dependencies)\n", "4. Before adding a new dependency: \n", " - Contact a [repository admin](https://github.com/CLIMADA-project/climada_python/people) to get permission\n", " - Open an [issue](https://github.com/CLIMADA-project/climada_python/issues)\n", " \n", "Hence, first try to solve your problem with the standard library and function/methods already implemented in CLIMADA (see in particular the [util functions](#id)) then use the packages included in CLIMADA, and if this is not enough, propose the addition of a new package. Do not hesitate to propose new packages if this is needed for your work!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Class inheritance " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In Python, a [class can inherit from other classes](https://docs.python.org/3/tutorial/classes.html), which is a very useful mechanism in certain circumstantce. However, it is wise to think about inheritance before implementing it. Very important, is that CLIMADA classes do not inherit from external library classes. For example, Exposure directly inherited from Geopandas. This caused problems in CLIMADA when the package Geopandas was updated.\n", "\n", "**CLIMADA classes shall NOT inherit classes from external modules**" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Does it belong into CLIMADA? " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "When developing for CLIMADA, it is important to distinguish between core content and particular applications. Core content is meant to be included into the [climada_python](https://github.com/CLIMADA-project/climada_python) repository and will be subject to a code review. Any new addition should first be discussed with one of the [repository admins](https://github.com/CLIMADA-project/climada_python/people). The purpose of this discussion is to see\n", "\n", "- How does the planed module fit into CLIMADA?\n", "- What is an optimal architecture for the new module?\n", "- What parts might already exist in other parts of the code?\n", "\n", "Applications made with CLIMADA, such as an [ECA study](https://eca-network.org/) can be stored in the [paper repository](https://github.com/CLIMADA-project/climada_papers) once they have been published. For other types of work, consider making a separate repository that imports CLIMADA as an external package." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Paper repository " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Applications made with CLIMADA which are published in the form of a paper or a report are very much encouraged to be submited to the [climada/paper](https://github.com/CLIMADA-project/climada_papers) repository. You can either:\n", "\n", "- Prepare a well-commented jupyter notebook with the code necessary to reproduce your results and upload it to the [climada/paper](https://github.com/CLIMADA-project/climada_papers) repository. Note however that the repository cannot be used for storing data files.\n", "- Upload the code necessary to reproduce your results to a separate repository of your own. Then, add a link to your repository and to your publication to the readme file on the [climada/paper](https://github.com/CLIMADA-project/climada_papers) repository.\n", "\n", "**Notes about DOI**\n", "\n", "Some journals requires you to provide a DOI to the code and data used for your publication. In this case, we encourage to create a separate repository for your code and create a DOI using [Zenodo](https://zenodo.org/) or any specific service from your institution (e.g. [ETH Zürich](https://documentation.library.ethz.ch/display/DOID/DOI+Registration+Manual)).\n", "\n", "The CLIMADA releases are also identified with a DOI." ] }, { "cell_type": "markdown", "metadata": { "ExecuteTime": { "end_time": "2021-01-15T07:55:22.871166Z", "start_time": "2021-01-15T07:55:22.865903Z" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Utility function" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "In CLIMADA, there is a set of utility functions defined in `climada.util`. A few examples are:\n", "\n", "- convert large monetary numbers into thousands, millions or billions together with the correct unit name\n", "- compute distances\n", "- load hdf5 files\n", "- convert iso country numbers between formats\n", "- ...\n", "\n", "Whenever you develop a module or make a code review, be attentive to see whether a given functionnality has already been implemented as a utility function. In addition, think carefully whether a given function/method does belong in its module or is actually independent of any particular module and should be defined as a utility function.\n", "\n", "It is very important to not reinvent the wheel and to avoid unnecessary redundancies in the code. This makes maintenance and debugging very tedious.\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Impact function renaming - if to impf " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "In the original CLIMADA code, the impact function is often referred to as `if` or `if_`. This is easy to confuse with the conditional operator *if*. Hence, in future a transition from\n", "\n", "`if` ---------> `impf`\n", "\n", "will be performed. Once the change is active, known developers will be notified and this message updated." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data dependencies" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Web APIs\n", "CLIMADA relies on open data available through web APIs such as those of the World Bank, Natural Earth, NASA and NOAA.\n", "You might execute the test ``climada_python-x.y.z/test_data_api.py`` to check that all the APIs used are active.\n", "If any is out of service (temporarily or permanently), the test will indicate which one.\n", "\n", "## Manual download\n", "\n", "As indicated in the software and tutorials, other data might need to be downloaded manually by the user. The following table shows these last data sources, their version used, its current availabilty and where they are used within CLIMADA:\n", "\n", "| Availability | Name | Version | Link | CLIMADA class | CLIMADA version | CLIMADA tutorial reference |\n", "|--------------|-------------------------------------------------|---------|------|---------------|-----------------|-------------------------------|\n", "| OK | Fire Information for Resource Management System | |[FIRMS](https://firms.modaps.eosdis.nasa.gov/download/) | BushFire | >V1.2.5 | climada_hazard_BushFire.ipynb |\n", "| OK | Gridded Population of the World (GPW) | v4.11 |[GPW4.11](https://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-count-rev11) | LitPop | > v1.2.3 | climada_entity_LitPop.ipynb |\n", "| FAILED | Gridded Population of the World (GPW) | v4.10 |[GPW1.10](https://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-count-rev10) | LitPop | >= v1.2.0 | climada_entity_LitPop.ipynb |\n", "| | | | | | | |" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Side Note on Parameters" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Don't use *args and **kwargs parameters without a very good reason.\n", "\n", "There _are_ valid use cases for [this kind of parameter notation](https://realpython.com/python-kwargs-and-args/).\\\n", "In particular `*args` comes in handy when there is an unknown number of equal typed arguments to be passed. E.g., the `pathlib.Path` constructor.\\\n", "But if the parameters are expected to be structured in any way, it is just a bad idea." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1 2 6\n" ] } ], "source": [ "def f(x, y, z):\n", " return x + y + z\n", "\n", "# bad in most cases\n", "def g(*args, **kwargs):\n", " x = args[0]\n", " y = kwargs['y']\n", " s = f(*args, **kwargs)\n", " print(x, y, s)\n", "\n", "g(1,y=2,z=3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# usually just fine\n", "def g(x, y, z):\n", " s = f(x, y, z)\n", " print(x, y, s)\n", "\n", "g(1,y=2,z=3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Decrease the number of parameters.\n", "\n", "Though CLIMADA's pylint configuration .pylintrc allows 7 arguments for any method or function before it complains, it is advisable to aim for less.\n", "It is quite likely that a function with so many parameters has an inherent design flaw.\\\n", "\n", "There are very well designed command line tools with innumerable optional arguments, e.g., rsync - but these are command line tools. There are also methods like pandas.DataFrame.plot() with countless optional arguments and it makes perfectly sense.\n", "\n", "But within the climada package it probably doesn't.\n", "divide et impera!\n", "\n", "Whenever a method has more than 5 parameters, it is more than likely that it can be refactored pretty easily into two or more methods with less parameters and less complexity:\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "f does many things with a lot of arguments: (1, 2, 3, 4, 5, 6, 7, 8)\n" ] }, { "data": { "text/plain": [ "36" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def f(a, b, c, d, e, f, g, h):\n", " print(f'f does many things with a lot of arguments: {a, b, c, d, e, f, g, h}')\n", " return sum([a, b, c, d, e, f, g, h])\n", " \n", "f(1, 2, 3, 4, 5, 6, 7, 8)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "f1 does less things with fewer arguments: (1, 2, 3, 4)\n", "f2 dito: (5, 6, 7, 8)\n", "f3 dito, but on a higher level: (10, 26)\n" ] }, { "data": { "text/plain": [ "36" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def f1(a, b, c, d):\n", " print(f'f1 does less things with fewer arguments: {a, b, c, d}')\n", " return sum([a, b, c, d])\n", "\n", "def f2(e, f, g, h):\n", " print(f'f2 dito: {e, f, g, h}')\n", " return sum([e, f, g, h])\n", "\n", "def f3(x, y):\n", " print(f'f3 dito, but on a higher level: {x, y}')\n", " return sum([x, y])\n", "\n", "f3(f1(1, 2, 3, 4), f2(5, 6, 7, 8))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This of course pleads the case on a strictly formal level. No real complexities have been reduced during the making of this example.\\\n", "Nevertheless there is the benefit of reduced test case requirements. And in real life real complexity _will_ be reduced." ] } ], "metadata": { "celltoolbar": "Slideshow", "hide_input": false, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.8" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autoclose": false, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": { "height": "625.2px", "left": "390px", "top": "110.117px", "width": "316.8px" }, "toc_section_display": true, "toc_window_display": true } }, "nbformat": 4, "nbformat_minor": 4 }