climada.entity.exposures.litpop package

climada.entity.exposures.litpop.gpw_population module

climada.entity.exposures.litpop.gpw_population.load_gpw_pop_shape(geometry, reference_year, gpw_version, data_dir=PosixPath('/home/docs/climada/data'), layer=0, verbose=True)

Read gridded population data from TIFF and crop to given shape(s).

Note: A (free) NASA Earthdata login is necessary to download the data. Data can be downloaded e.g. for gpw_version=11 and year 2015 from https://sedac.ciesin.columbia.edu/downloads/data/gpw-v4/ gpw-v4-population-count-rev11/gpw-v4-population-count-rev11_2015_30_sec_tif.zip

Parameters:

• geometry (shape(s) to crop data to in degree lon/lat.) – for example shapely.geometry.(Multi)Polygon or shapefile.Shape from polygon(s) defined in a (country) shapefile.

• reference_year (int) – target year for data extraction

• gpw_version (int) – Version number of GPW population data, i.e. 11 for v4.11. The default is CONFIG.exposures.litpop.gpw_population.gpw_version.int()

• data_dir (Path, optional) – Path to data directory holding GPW data folders. The default is SYSTEM_DIR.

• layer (int, optional) – relevant data layer in input TIFF file to return. The default is 0 and should not be changed without understanding the different data layers in the given TIFF file.

• verbose (bool, optional) – Enable verbose logging about the used GPW version and reference year. Default: True.

Returns:

• pop_data (2D numpy array) – contains extracted population count data per grid point in shape first dimension is lat, second dimension is lon.

• meta (dict) – contains meta data per array, including “transform” with meta data on coordinates.

• global_transform (Affine instance) – contains six numbers, providing transform info for global GWP grid. global_transform is required for resampling on a globally consistent grid

climada.entity.exposures.litpop.gpw_population.get_gpw_file_path(gpw_version, reference_year, data_dir=None, verbose=True)

Check available GPW population data versions and year closest to reference_year and return full path to TIFF file.

Parameters:

• gpw_version (int (optional)) – Version number of GPW population data, i.e. 11 for v4.11.

• reference_year (int (optional)) – Data year is selected as close to reference_year as possible. The default is 2020.

• data_dir (pathlib.Path (optional)) – Absolute path where files are stored. Default: SYSTEM_DIR

• verbose (bool, optional) – Enable verbose logging about the used GPW version and reference year. Default: True.

Raises:

FileExistsError –

Returns:

pathlib.Path

Return type:

path to input file with population data

climada.entity.exposures.litpop.litpop module

climada.entity.exposures.litpop.litpop.GPW_VERSION = 11

Version of Gridded Population of the World (GPW) input data. Check for updates.

class climada.entity.exposures.litpop.litpop.LitPop(*args, meta=None, description=None, ref_year=2018, value_unit='USD', crs=None, **kwargs)

Bases: Exposures

Holds geopandas GeoDataFrame with metada and columns (pd.Series) defined in Attributes of Exposures class. LitPop exposure values are disaggregated proportional to a combination of nightlight intensity (NASA) and Gridded Population data (SEDAC). Total asset values can be produced capital, population count, GDP, or non-financial wealth.

Calling sequence example: country_names = [‘CHE’, ‘Austria’] exp = LitPop.from_countries(country_names) exp.plot()

exponents

Defining powers (m, n) with which lit (nightlights) and pop (gpw) go into Lit**m * Pop**n. The default is (1,1).

Type:

tuple of two integers, optional

fin_mode

Socio-economic value to be used as an asset base that is disaggregated. The default is ‘pc’.

Type:

str, optional

gpw_version

Version number of GPW population data, e.g. 11 for v4.11. The default is defined in GPW_VERSION.

Type:

int, optional

set_countries(*args, **kwargs)

This function is deprecated, use LitPop.from_countries instead.

classmethod from_countries(countries, res_arcsec=30, exponents=(1, 1), fin_mode='pc', total_values=None, admin1_calc=False, reference_year=2018, gpw_version=11, data_dir=PosixPath('/home/docs/climada/data'))

Init new LitPop exposure object for a list of countries (admin 0).

Sets attributes ref_year, crs, value, geometry, meta, value_unit, exponents,`fin_mode`, gpw_version, and admin1_calc.

Parameters:

• countries (list with str or int) – list containing country identifiers: iso3alpha (e.g. ‘JPN’), iso3num (e.g. 92) or name (e.g. ‘Togo’)

• res_arcsec (float, optional) – Horizontal resolution in arc-sec. The default is 30 arcsec, this corresponds to roughly 1 km.

• exponents (tuple of two integers, optional) – Defining power with which lit (nightlights) and pop (gpw) go into LitPop. To get nightlights^3 without population count: (3, 0). To use population count alone: (0, 1). Default: (1, 1)

• fin_mode (str, optional) – Socio-economic value to be used as an asset base that is disaggregated to the grid points within the country:

  • ‘pc’: produced capital (Source: World Bank), incl. manufactured or built assets such as machinery, equipment, and physical structures pc is in constant 2014 USD.

  • ‘pop’: population count (source: GPW, same as gridded population). The unit is ‘people’.

  • ‘gdp’: gross-domestic product (Source: World Bank) [USD]

  • ‘income_group’: gdp multiplied by country’s income group+1 [USD]. Income groups are 1 (low) to 4 (high income).

  • ‘nfw’: non-financial wealth (Source: Credit Suisse, of households only) [USD]

  • ‘tw’: total wealth (Source: Credit Suisse, of households only) [USD]

  • ‘norm’: normalized by country (no unit)

  • ‘none’: LitPop per pixel is returned unchanged (no unit)

  Default: ‘pc’

• total_values (list containing numerics, same length as countries, optional) – Total values to be disaggregated to grid in each country. The default is None. If None, the total number is extracted from other sources depending on the value of fin_mode.

• admin1_calc (boolean, optional) – If True, distribute admin1-level GDP (if available). Default: False

• reference_year (int, optional) – Reference year. Default: CONFIG.exposures.def_ref_year.

• gpw_version (int, optional) – Version number of GPW population data. The default is GPW_VERSION

• data_dir (Path, optional) – redefines path to input data directory. The default is SYSTEM_DIR.

Raises:

ValueError –

Returns:

exp – LitPop instance with exposure for given countries

Return type:

LitPop

set_nightlight_intensity(*args, **kwargs)

This function is deprecated, use LitPop.from_nightlight_intensity instead.

classmethod from_nightlight_intensity(countries=None, shape=None, res_arcsec=15, reference_year=2018, data_dir=PosixPath('/home/docs/climada/data'))

Wrapper around from_countries / from_shape.

Initiate exposures instance with value equal to the original BlackMarble nightlight intensity resampled to the target resolution res_arcsec.

Provide either countries or shape.

Parameters:

• countries (list or str, optional) – list containing country identifiers (name or iso3)

• shape (Shape, Polygon or MultiPolygon, optional) – geographical shape of target region, alternative to countries.

• res_arcsec (int, optional) – Resolution in arc seconds. The default is 15.

• reference_year (int, optional) – Reference year. The default is CONFIG.exposures.def_ref_year.

• data_dir (Path, optional) – data directory. The default is None.

Raises:

ValueError –

Returns:

exp – Exposure instance with values representing pure nightlight intensity from input nightlight data (BlackMarble)

Return type:

LitPop

set_population(*args, **kwargs)

This function is deprecated, use LitPop.from_population instead.

classmethod from_population(countries=None, shape=None, res_arcsec=30, reference_year=2018, gpw_version=11, data_dir=PosixPath('/home/docs/climada/data'))

Wrapper around from_countries / from_shape.

Initiate exposures instance with value equal to GPW population count. Provide either countries or shape.

Parameters:

• countries (list or str, optional) – list containing country identifiers (name or iso3)

• shape (Shape, Polygon or MultiPolygon, optional) – geographical shape of target region, alternative to countries.

• res_arcsec (int, optional) – Resolution in arc seconds. The default is 30.

• reference_year (int, optional) – Reference year (closest available GPW data year is used) The default is CONFIG.exposures.def_ref_year.

• gpw_version (int, optional) – specify GPW data verison. The default is 11.

• data_dir (Path, optional) – data directory. The default is None. Either countries or shape is required.

Raises:

ValueError –

Returns:

exp – Exposure instance with values representing population count according to Gridded Population of the World (GPW) input data set.

Return type:

LitPop

set_custom_shape_from_countries(*args, **kwargs)

This function is deprecated, use LitPop.from_shape_and_countries instead.

classmethod from_shape_and_countries(shape, countries, res_arcsec=30, exponents=(1, 1), fin_mode='pc', admin1_calc=False, reference_year=2018, gpw_version=11, data_dir=PosixPath('/home/docs/climada/data'))

create LitPop exposure for country and then crop to given shape.

Parameters:

• shape (shapely.geometry.Polygon, MultiPolygon, shapereader.Shape,) – or GeoSeries or list containg either Polygons or Multipolygons. Geographical shape for which LitPop Exposure is to be initiated.

• countries (list with str or int) – list containing country identifiers: iso3alpha (e.g. ‘JPN’), iso3num (e.g. 92) or name (e.g. ‘Togo’)

• res_arcsec (float, optional) – Horizontal resolution in arc-sec. The default is 30 arcsec, this corresponds to roughly 1 km.

• exponents (tuple of two integers, optional) – Defining power with which lit (nightlights) and pop (gpw) go into LitPop. Default: (1, 1)

• fin_mode (str, optional) – Socio-economic value to be used as an asset base that is disaggregated to the grid points within the country:

  • ‘pc’: produced capital (Source: World Bank), incl. manufactured or built assets such as machinery, equipment, and physical structures (pc is in constant 2014 USD)

  • ‘pop’: population count (source: GPW, same as gridded population). The unit is ‘people’.

  • ‘gdp’: gross-domestic product (Source: World Bank) [USD]

  • ‘income_group’: gdp multiplied by country’s income group+1 [USD] Income groups are 1 (low) to 4 (high income).

  • ‘nfw’: non-financial wealth (Source: Credit Suisse, of households only) [USD]

  • ‘tw’: total wealth (Source: Credit Suisse, of households only) [USD]

  • ‘norm’: normalized by country

  • ‘none’: LitPop per pixel is returned unchanged

  Default: ‘pc’

• admin1_calc (boolean, optional) – If True, distribute admin1-level GDP (if available). Default: False

• reference_year (int, optional) – Reference year for data sources. Default: 2020

• gpw_version (int, optional) – Version number of GPW population data. The default is GPW_VERSION

• data_dir (Path, optional) – redefines path to input data directory. The default is SYSTEM_DIR.

Raises:

NotImplementedError –

Returns:

exp – The exposure LitPop within shape

Return type:

LitPop

set_custom_shape(*args, **kwargs)

This function is deprecated, use LitPop.from_shape instead.

classmethod from_shape(shape, total_value, res_arcsec=30, exponents=(1, 1), value_unit='USD', region_id=None, reference_year=2018, gpw_version=11, data_dir=PosixPath('/home/docs/climada/data'))

init LitPop exposure object for a custom shape. Requires user input regarding the total value to be disaggregated.

Sets attributes ref_year, crs, value, geometry, meta, value_unit, exponents,`fin_mode`, gpw_version, and admin1_calc.

This method can be used to initiated LitPop Exposure for sub-national regions such as states, districts, cantons, cities, … but shapes and total value need to be provided manually. If these required input parameters are not known / available, better initiate Exposure for entire country and extract shape afterwards.

Parameters:

• shape (shapely.geometry.Polygon or MultiPolygon or shapereader.Shape.) – Geographical shape for which LitPop Exposure is to be initiated.

• total_value (int, float or None type) – Total value to be disaggregated to grid in shape. If None, no value is disaggregated.

• res_arcsec (float, optional) – Horizontal resolution in arc-sec. The default 30 arcsec corresponds to roughly 1 km.

• exponents (tuple of two integers, optional) – Defining power with which lit (nightlights) and pop (gpw) go into LitPop.

• value_unit (str) – Unit of exposure values. The default is USD.

• region_id (int, optional) – The numeric ISO 3166 region associated with the shape. If set to a value, this single value will be set for every coordinate in the GeoDataFrame of the resulting LitPop instance. If None (default), the region ID for every coordinate will be determined automatically (at a slight computational cost).

• reference_year (int, optional) – Reference year for data sources. Default: CONFIG.exposures.def_ref_year

• gpw_version (int, optional) – Version number of GPW population data. The default is set in CONFIG.

• data_dir (Path, optional) – redefines path to input data directory. The default is SYSTEM_DIR.

Raises:

• NotImplementedError –

• ValueError –

• TypeError –

Returns:

exp – The exposure LitPop within shape

Return type:

LitPop

set_country(*args, **kwargs)

This function is deprecated, use LitPop.from_countries instead.

climada.entity.exposures.litpop.litpop.get_value_unit(fin_mode)

get value_unit depending on fin_mode

Parameters:

fin_mode (Socio-economic value to be used as an asset base)

Returns:

value_unit

Return type:

str

climada.entity.exposures.litpop.litpop.reproject_input_data(data_array_list, meta_list, i_align=0, target_res_arcsec=None, global_origins=(-180.0, 89.99999999999991), resampling=Resampling.bilinear, conserve=None)

LitPop-sepcific wrapper around u_coord.align_raster_data.

Reprojects all arrays in data_arrays to a given resolution – all based on the population data grid.

Parameters:

• data_array_list (list or array of numpy arrays containing numbers) – Data to be reprojected, i.e. list containing N (min. 1) 2D-arrays. The data with the reference grid used to align the global destination grid to should be first data_array_list[i_align], e.g., pop (GPW population data) for LitPop.

• meta_list (list of dicts) – meta data dictionaries of data arrays in same order as data_array_list. Required fields in each dict are ‘dtype,’, ‘width’, ‘height’, ‘crs’, ‘transform’. Example:

  >>> {
  ...     'driver': 'GTiff',
  ...     'dtype': 'float32',
  ...     'nodata': 0,
  ...     'width': 2702,
  ...     'height': 1939,
  ...     'count': 1,
  ...     'crs': CRS.from_epsg(4326),
  ...     'transform': Affine(0.00833333333333333, 0.0, -18.175000000000068,
  ...                         0.0, -0.00833333333333333, 43.79999999999993),
  ... }
  

  The meta data with the reference grid used to define the global destination grid should be first in the list, e.g., GPW population data for LitPop.

• i_align (int, optional) – Index/Position of meta in meta_list to which the global grid of the destination is to be aligned to (c.f. u_coord.align_raster_data) The default is 0.

• target_res_arcsec (int, optional) – target resolution in arcsec. The default is None, i.e. same resolution as reference data.

• global_origins (tuple with two numbers (lat, lon), optional) – global lon and lat origins as basis for destination grid. The default is the same as for GPW population data: (-180.0, 89.99999999999991)

• resampling (resampling function, optional) – The default is rasterio.warp.Resampling.bilinear

• conserve (str, optional, either ‘mean’ or ‘sum’) – Conserve mean or sum of data? The default is None (no conservation).

Returns:

• data_array_list (list) – contains reprojected data sets

• meta_out (dict) – contains meta data of new grid (same for all arrays)

climada.entity.exposures.litpop.litpop.gridpoints_core_calc(data_arrays, offsets=None, exponents=None, total_val_rescale=None)

Combines N dense numerical arrays by point-wise multipilcation and optionally rescales to new total value:

1. An offset (1 number per array) is added to all elements in the corresponding data array in data_arrays (optional).

2. Numbers in each array are taken to the power of the corresponding exponent (optional).

3. Arrays are multiplied element-wise.

4. if total_val_rescale is provided, results are normalized and re-scaled with total_val_rescale.

5. One array with results is returned.

Parameters:

• data_arrays (list or array of numpy arrays containing numbers) – Data to be combined, i.e. list containing N (min. 1) arrays of same shape.

• total_val_rescale (float or int, optional) – Total value for optional rescaling of resulting array. All values in result_array are skaled so that the sum is equal to total_val_rescale. The default (None) implies no rescaling.

• offsets (list or array containing N numbers >= 0, optional) – One numerical offset per array that is added (sum) to the corresponding array in data_arrays. The default (None) corresponds to np.zeros(N).

• exponents (list or array containing N numbers >= 0, optional) – One exponent per array used as power for the corresponding array. The default (None) corresponds to np.ones(N).

Raises:

ValueError – If input lists don’t have the same number of elements. Or: If arrays in data_arrays do not have the same shape.

Returns:

Results from calculation described above.

Return type:

np.array of same shape as arrays in data_arrays

climada.entity.exposures.litpop.nightlight module

climada.entity.exposures.litpop.nightlight.NOAA_RESOLUTION_DEG = 0.008333333333333333

NOAA nightlights coordinates resolution in degrees.

climada.entity.exposures.litpop.nightlight.NASA_RESOLUTION_DEG = 0.004166666666666667

NASA nightlights coordinates resolution in degrees.

climada.entity.exposures.litpop.nightlight.NASA_TILE_SIZE = (21600, 21600)

NASA nightlights tile resolution.

climada.entity.exposures.litpop.nightlight.NOAA_BORDER = (-180, -65, 180, 75)

NOAA nightlights border (min_lon, min_lat, max_lon, max_lat)

climada.entity.exposures.litpop.nightlight.BM_FILENAMES = ['BlackMarble_%i_A1_geo_gray.tif', 'BlackMarble_%i_A2_geo_gray.tif', 'BlackMarble_%i_B1_geo_gray.tif', 'BlackMarble_%i_B2_geo_gray.tif', 'BlackMarble_%i_C1_geo_gray.tif', 'BlackMarble_%i_C2_geo_gray.tif', 'BlackMarble_%i_D1_geo_gray.tif', 'BlackMarble_%i_D2_geo_gray.tif']

Nightlight NASA files which generate the whole earth when put together.

climada.entity.exposures.litpop.nightlight.load_nasa_nl_shape(geometry, year, data_dir=PosixPath('/home/docs/climada/data'), dtype='float32')

Read nightlight data from NASA BlackMarble tiles cropped to given shape(s) and combine arrays from each tile.

1. check and download required blackmarble files

2. read and crop data from each file required in a bounding box around the given geometry.

3. combine data from all input files into one array. this array then contains all data in the geographic bounding box around geometry.

4. return array with nightlight data

Parameters:

• geometry (shape(s) to crop data to in degree lon/lat.) – for example shapely.geometry.(Multi)Polygon or shapefile.Shape. from polygon defined in a shapefile. The object should have attribute ‘bounds’ or ‘points’

• year (int) – target year for nightlight data, e.g. 2016. Closest availble year is selected.

• data_dir (Path (optional)) – Path to directory with BlackMarble data. The default is SYSTEM_DIR.

• dtype (dtype) – data type for output default ‘float32’, required for LitPop, choose ‘int8’ for integer.

Returns:

• results_array (numpy array) – extracted and combined nightlight data for bounding box around shape

• meta (dict) – rasterio meta data for results_array

climada.entity.exposures.litpop.nightlight.get_required_nl_files(bounds)

Determines which of the satellite pictures are necessary for

a certain bounding box (e.g. country)

Parameters:

bounds (1x4 tuple) – bounding box from shape (min_lon, min_lat, max_lon, max_lat).

Raises:

ValueError – invalid bounds

Returns:

req_files – Array indicating the required files for the current operation with a boolean value (1: file is required, 0: file is not required).

Return type:

numpy array

climada.entity.exposures.litpop.nightlight.check_nl_local_file_exists(required_files=None, check_path=PosixPath('/home/docs/climada/data'), year=2016)

Checks if BM Satellite files are avaialbe and returns a vector denoting the missing files.

Parameters:

• required_files (numpy array, optional) – boolean array of dimension (8,) with which some files can be skipped. Only files with value 1 are checked, with value zero are skipped. The default is np.ones(len(BM_FILENAMES),)

• check_path (str or Path) – absolute path where files are stored. Default: SYSTEM_DIR

• year (int) – year of the image, e.g. 2016

Returns:

files_exist – Boolean array that denotes if the required files exist.

Return type:

numpy array

climada.entity.exposures.litpop.nightlight.download_nl_files(req_files=array([1., 1., 1., 1., 1., 1., 1., 1.]), files_exist=array([0., 0., 0., 0., 0., 0., 0., 0.]), dwnl_path=PosixPath('/home/docs/climada/data'), year=2016)

Attempts to download nightlight files from NASA webpage.

Parameters:

• req_files (numpy array, optional) –

  Boolean array which indicates the files required (0-> skip, 1-> download).

  The default is np.ones(len(BM_FILENAMES),).

• files_exist (numpy array, optional) –

  Boolean array which indicates if the files already

  exist locally and should not be downloaded (0-> download, 1-> skip). The default is np.zeros(len(BM_FILENAMES),).

• dwnl_path (str or path, optional) – Download directory path. The default is SYSTEM_DIR.

• year (int, optional) – Data year to be downloaded. The default is 2016.

Raises:

• ValueError –

• RuntimeError –

Returns:

dwnl_path – Download directory path.

Return type:

str or path

climada.entity.exposures.litpop.nightlight.load_nasa_nl_shape_single_tile(geometry, path, layer=0)

Read nightlight data from single NASA BlackMarble tile and crop to given shape.

Parameters:

• geometry (shape or geometry object) – shape(s) to crop data to in degree lon/lat. for example shapely.geometry.Polygon object or from polygon defined in a shapefile.

• path (Path or str) – full path to BlackMarble tif (including filename)

• layer (int, optional) – TIFF-layer to be returned. The default is 0. BlackMarble usually comes with 3 layers.

Returns:

• out_image[layer, (,:] : 2D numpy ndarray) – 2d array with data cropped to bounding box of shape

• meta (dict) – rasterio meta

climada.entity.exposures.litpop.nightlight.load_nightlight_nasa(bounds, req_files, year)

Get nightlight from NASA repository that contain input boundary.

Note: Legacy for BlackMarble, not required for litpop module

Parameters:

• bounds (tuple) – min_lon, min_lat, max_lon, max_lat

• req_files (np.array) – array with flags for NASA files needed

• year (int) – nightlight year

Returns:

• nightlight (sparse.csr_matrix)

• coord_nl (np.array)

climada.entity.exposures.litpop.nightlight.read_bm_file(bm_path, filename)

Reads a single NASA BlackMarble GeoTiff and returns the data. Run all required checks first.

Note: Legacy for BlackMarble, not required for litpop module

Parameters:

• bm_path (str) – absolute path where files are stored.

• filename (str) – filename of the file to be read.

Returns:

• arr1 (array) – Raw BM data

• curr_file (gdal GeoTiff File) – Additional info from which coordinates can be calculated.

climada.entity.exposures.litpop.nightlight.unzip_tif_to_py(file_gz)

Unzip image file, read it, flip the x axis, save values as pickle and remove tif.

Parameters:

file_gz (str) – file fith .gz format to unzip

Returns:

• fname (str) – file_name of unzipped file

• nightlight (sparse.csr_matrix)

climada.entity.exposures.litpop.nightlight.untar_noaa_stable_nightlight(f_tar_ini)

Move input tar file to SYSTEM_DIR and extract stable light file. Returns absolute path of stable light file in format tif.gz.

Parameters:

f_tar_ini (str) – absolute path of file

Returns:

f_tif_gz – path of stable light file

Return type:

str

climada.entity.exposures.litpop.nightlight.load_nightlight_noaa(ref_year=2013, sat_name=None)

Get nightlight luminosites. Nightlight matrix, lat and lon ordered such that nightlight[1][0] corresponds to lat[1], lon[0] point (the image has been flipped).

Parameters:

• ref_year (int, optional) – reference year. The default is 2013.

• sat_name (str, optional) – satellite provider (e.g. ‘F10’, ‘F18’, …)

Returns:

• nightlight (sparse.csr_matrix)

• coord_nl (np.array)

• fn_light (str)