climada.entity.exposures.litpop package¶
climada.entity.exposures.litpop.gpw_population module¶
- climada.entity.exposures.litpop.gpw_population.load_gpw_pop_shape(geometry, reference_year, gpw_version, data_dir=PosixPath('/home/docs/climada/data'), layer=0, verbatim=True)[source]¶
Read gridded population data from TIFF and crop to given shape(s).
Note: A (free) NASA Earthdata login is necessary to download the data. Data can be downloaded e.g. for gpw_version=11 from https://sedac.ciesin.columbia.edu/downloads/data/gpw-v4/ gpw-v4-population-count-rev11/gpw-v4-population-count-rev11_2015_30_sec_tif.zip
- Parameters
geometry (shape(s) to crop data to in degree lon/lat.) – for example shapely.geometry.(Multi)Polygon or shapefile.Shape from polygon(s) defined in a (country) shapefile.
reference_year (int) – target year for data extraction
gpw_version (int) – Version number of GPW population data, i.e. 11 for v4.11. The default is CONFIG.exposures.litpop.gpw_population.gpw_version.int()
data_dir (Path, optional) – Path to data directory holding GPW data folders. The default is SYSTEM_DIR.
layer (int, optional) – relevant data layer in input TIFF file to return. The default is 0 and should not be changed without understanding the different data layers in the given TIFF file.
verbatim (bool (optional):) – if False, output in LOGGER is surpressed. Default is True.
- Returns
pop_data (2D numpy array) – contains extracted population count data per grid point in shape first dimension is lat, second dimension is lon.
meta (dict) – contains meta data per array, including “transform” with meta data on coordinates.
global_transform (Affine instance) – contains six numbers, providing transform info for global GWP grid. global_transform is required for resampling on a globally consistent grid
- climada.entity.exposures.litpop.gpw_population.get_gpw_file_path(gpw_version, reference_year, data_dir=PosixPath('/home/docs/climada/data'), verbatim=True)[source]¶
Check available GPW population data versions and year closest to reference_year and return full path to TIFF file.
- Parameters
gpw_version (int (optional)) – Version number of GPW population data, i.e. 11 for v4.11.
reference_year (int (optional)) – Data year is selected as close to reference_year as possible. The default is 2020.
data_dir (pathlib.Path (optional)) – Absolute path where files are stored. Default: SYSTEM_DIR
- Raises
FileExistsError –
- Returns
pathlib.Path
- Return type
path to input file with population data
climada.entity.exposures.litpop.litpop module¶
- climada.entity.exposures.litpop.litpop.GPW_VERSION = 11¶
Version of Gridded Population of the World (GPW) input data. Check for updates.
- class climada.entity.exposures.litpop.litpop.LitPop(*args, meta=None, tag=None, ref_year=2018, value_unit='USD', crs=None, **kwargs)[source]¶
Bases:
climada.entity.exposures.base.Exposures
Holds geopandas GeoDataFrame with metada and columns (pd.Series) defined in Attributes of Exposures class. LitPop exposure values are disaggregated proportional to a combination of nightlight intensity (NASA) and Gridded Population data (SEDAC). Total asset values can be produced capital, population count, GDP, or non-financial wealth.
Calling sequence example: country_names = [‘CHE’, ‘Austria’] exp = LitPop.from_countries(country_names) exp.plot()
- exponents¶
Defining powers (m, n) with which lit (nightlights) and pop (gpw) go into Lit**m * Pop**n. The default is (1,1).
- Type
tuple of two integers, optional
- fin_mode¶
Socio-economic value to be used as an asset base that is disaggregated. The default is ‘pc’.
- Type
str, optional
- gpw_version¶
Version number of GPW population data, e.g. 11 for v4.11. The default is defined in GPW_VERSION.
- Type
int, optional
- set_countries(*args, **kwargs)[source]¶
This function is deprecated, use LitPop.from_countries instead.
- classmethod from_countries(countries, res_arcsec=30, exponents=(1, 1), fin_mode='pc', total_values=None, admin1_calc=False, reference_year=2018, gpw_version=11, data_dir=PosixPath('/home/docs/climada/data'))[source]¶
Init new LitPop exposure object for a list of countries (admin 0). Sets attributes ref_year, tag, crs, value, geometry, meta, value_unit, exponents,`fin_mode`, gpw_version, and admin1_calc.
- Parameters
countries (list with str or int) – list containing country identifiers: iso3alpha (e.g. ‘JPN’), iso3num (e.g. 92) or name (e.g. ‘Togo’)
res_arcsec (float, optional) – Horizontal resolution in arc-sec. The default is 30 arcsec, this corresponds to roughly 1 km.
exponents (tuple of two integers, optional) – Defining power with which lit (nightlights) and pop (gpw) go into LitPop. To get nightlights^3 without population count: (3, 0). To use population count alone: (0, 1). Default: (1, 1)
fin_mode (str, optional) – Socio-economic value to be used as an asset base that is disaggregated to the grid points within the country
‘pc’: produced capital (Source: World Bank), incl. manufactured or built assets such as machinery, equipment, and physical structures pc is in constant 2014 USD.
‘pop’: population count (source: GPW, same as gridded population). The unit is ‘people’.
‘gdp’: gross-domestic product (Source: World Bank) [USD]
‘income_group’: gdp multiplied by country’s income group+1 [USD]. Income groups are 1 (low) to 4 (high income).
‘nfw’: non-financial wealth (Source: Credit Suisse, of households only) [USD]
‘tw’: total wealth (Source: Credit Suisse, of households only) [USD]
‘norm’: normalized by country (no unit)
‘none’: LitPop per pixel is returned unchanged (no unit)
The default is ‘pc’.
total_values (list containing numerics, same length as countries, optional) – Total values to be disaggregated to grid in each country. The default is None. If None, the total number is extracted from other sources depending on the value of fin_mode.
admin1_calc (boolean, optional) – If True, distribute admin1-level GDP (if available). Default: False
reference_year (int, optional) – Reference year. Default: CONFIG.exposures.def_ref_year.
gpw_version (int, optional) – Version number of GPW population data. The default is GPW_VERSION
data_dir (Path, optional) – redefines path to input data directory. The default is SYSTEM_DIR.
- Raises
ValueError –
- Returns
exp – LitPop instance with exposure for given countries
- Return type
- set_nightlight_intensity(*args, **kwargs)[source]¶
This function is deprecated, use LitPop.from_nightlight_intensity instead.
- classmethod from_nightlight_intensity(countries=None, shape=None, res_arcsec=15, reference_year=2018, data_dir=PosixPath('/home/docs/climada/data'))[source]¶
Wrapper around from_countries / from_shape.
Initiate exposures instance with value equal to the original BlackMarble nightlight intensity resampled to the target resolution res_arcsec.
Provide either countries or shape.
- Parameters
countries (list or str, optional) – list containing country identifiers (name or iso3)
shape (Shape, Polygon or MultiPolygon, optional) – geographical shape of target region, alternative to countries.
res_arcsec (int, optional) – Resolution in arc seconds. The default is 15.
reference_year (int, optional) – Reference year. The default is CONFIG.exposures.def_ref_year.
data_dir (Path, optional) – data directory. The default is None.
- Raises
ValueError –
- Returns
exp – Exposure instance with values representing pure nightlight intensity from input nightlight data (BlackMarble)
- Return type
- set_population(*args, **kwargs)[source]¶
This function is deprecated, use LitPop.from_population instead.
- classmethod from_population(countries=None, shape=None, res_arcsec=30, reference_year=2018, gpw_version=11, data_dir=PosixPath('/home/docs/climada/data'))[source]¶
Wrapper around from_countries / from_shape.
Initiate exposures instance with value equal to GPW population count. Provide either countries or shape.
- Parameters
countries (list or str, optional) – list containing country identifiers (name or iso3)
shape (Shape, Polygon or MultiPolygon, optional) – geographical shape of target region, alternative to countries.
res_arcsec (int, optional) – Resolution in arc seconds. The default is 30.
reference_year (int, optional) – Reference year (closest available GPW data year is used) The default is CONFIG.exposures.def_ref_year.
gpw_version (int, optional) – specify GPW data verison. The default is 11.
data_dir (Path, optional) – data directory. The default is None. Either countries or shape is required.
- Raises
ValueError –
- Returns
exp – Exposure instance with values representing population count according to Gridded Population of the World (GPW) input data set.
- Return type
- set_custom_shape_from_countries(*args, **kwargs)[source]¶
This function is deprecated, use LitPop.from_shape_and_countries instead.
- classmethod from_shape_and_countries(shape, countries, res_arcsec=30, exponents=(1, 1), fin_mode='pc', admin1_calc=False, reference_year=2018, gpw_version=11, data_dir=PosixPath('/home/docs/climada/data'))[source]¶
create LitPop exposure for country and then crop to given shape.
- Parameters
shape (shapely.geometry.Polygon, MultiPolygon, shapereader.Shape,) – or GeoSeries or list containg either Polygons or Multipolygons. Geographical shape for which LitPop Exposure is to be initiated.
countries (list with str or int) – list containing country identifiers: iso3alpha (e.g. ‘JPN’), iso3num (e.g. 92) or name (e.g. ‘Togo’)
res_arcsec (float, optional) – Horizontal resolution in arc-sec. The default is 30 arcsec, this corresponds to roughly 1 km.
exponents (tuple of two integers, optional) – Defining power with which lit (nightlights) and pop (gpw) go into LitPop. Default: (1, 1)
fin_mode (str, optional) – Socio-economic value to be used as an asset base that is disaggregated to the grid points within the country
‘pc’: produced capital (Source: World Bank), incl. manufactured or built assets such as machinery, equipment, and physical structures (pc is in constant 2014 USD)
‘pop’: population count (source: GPW, same as gridded population). The unit is ‘people’.
‘gdp’: gross-domestic product (Source: World Bank) [USD]
‘income_group’: gdp multiplied by country’s income group+1 [USD] Income groups are 1 (low) to 4 (high income).
‘nfw’: non-financial wealth (Source: Credit Suisse, of households only) [USD]
‘tw’: total wealth (Source: Credit Suisse, of households only) [USD]
‘norm’: normalized by country
‘none’: LitPop per pixel is returned unchanged
The default is ‘pc’.
admin1_calc (boolean, optional) – If True, distribute admin1-level GDP (if available). Default: False
reference_year (int, optional) – Reference year for data sources. Default: 2020
gpw_version (int, optional) – Version number of GPW population data. The default is GPW_VERSION
data_dir (Path, optional) – redefines path to input data directory. The default is SYSTEM_DIR.
- Raises
NotImplementedError –
- Returns
exp – The exposure LitPop within shape
- Return type
- set_custom_shape(*args, **kwargs)[source]¶
This function is deprecated, use LitPop.from_shape instead.
- classmethod from_shape(shape, total_value, res_arcsec=30, exponents=(1, 1), value_unit='USD', reference_year=2018, gpw_version=11, data_dir=PosixPath('/home/docs/climada/data'))[source]¶
init LitPop exposure object for a custom shape. Requires user input regarding the total value to be disaggregated.
Sets attributes ref_year, tag, crs, value, geometry, meta, value_unit, exponents,`fin_mode`, gpw_version, and admin1_calc.
This method can be used to initiated LitPop Exposure for sub-national regions such as states, districts, cantons, cities, … but shapes and total value need to be provided manually. If these required input parameters are not known / available, better initiate Exposure for entire country and extract shape afterwards.
- Parameters
shape (shapely.geometry.Polygon or MultiPolygon or shapereader.Shape.) – Geographical shape for which LitPop Exposure is to be initiated.
total_value (int, float or None type) – Total value to be disaggregated to grid in shape. If None, no value is disaggregated.
res_arcsec (float, optional) – Horizontal resolution in arc-sec. The default 30 arcsec corresponds to roughly 1 km.
exponents (tuple of two integers, optional) – Defining power with which lit (nightlights) and pop (gpw) go into LitPop.
value_unit (str) – Unit of exposure values. The default is USD.
reference_year (int, optional) – Reference year for data sources. Default: CONFIG.exposures.def_ref_year
gpw_version (int, optional) – Version number of GPW population data. The default is set in CONFIG.
data_dir (Path, optional) – redefines path to input data directory. The default is SYSTEM_DIR.
- Raises
NotImplementedError –
ValueError –
TypeError –
- Returns
exp – The exposure LitPop within shape
- Return type
- set_country(*args, **kwargs)¶
This function is deprecated, use LitPop.from_countries instead.
- climada.entity.exposures.litpop.litpop.get_value_unit(fin_mode)[source]¶
get value_unit depending on fin_mode
- Parameters
fin_mode (Socio-economic value to be used as an asset base)
- Returns
value_unit
- Return type
str
- climada.entity.exposures.litpop.litpop.reproject_input_data(data_array_list, meta_list, i_align=0, target_res_arcsec=None, global_origins=(- 180.0, 89.99999999999991), resampling=Resampling.bilinear, conserve=None)[source]¶
LitPop-sepcific wrapper around u_coord.align_raster_data.
Reprojects all arrays in data_arrays to a given resolution – all based on the population data grid.
- Parameters
data_array_list (list or array of numpy arrays containing numbers) – Data to be reprojected, i.e. list containing N (min. 1) 2D-arrays. The data with the reference grid used to align the global destination grid to should be first data_array_list[i_align], e.g., pop (GPW population data) for LitPop.
meta_list (list of dicts) – meta data dictionaries of data arrays in same order as data_array_list. Required fields in each dict are ‘dtype,’, ‘width’, ‘height’, ‘crs’, ‘transform’. Example:
- {‘driver’: ‘GTiff’,
‘dtype’: ‘float32’, ‘nodata’: 0, ‘width’: 2702, ‘height’: 1939, ‘count’: 1, ‘crs’: CRS.from_epsg(4326), ‘transform’: Affine(0.00833333333333333, 0.0, -18.175000000000068,
0.0, -0.00833333333333333, 43.79999999999993)}
The meta data with the reference grid used to define the global destination grid should be first in the list, e.g., GPW population data for LitPop.
i_align (int, optional) – Index/Position of meta in meta_list to which the global grid of the destination is to be aligned to (c.f. u_coord.align_raster_data) The default is 0.
target_res_arcsec (int, optional) – target resolution in arcsec. The default is None, i.e. same resolution as reference data.
global_origins (tuple with two numbers (lat, lon), optional) – global lon and lat origins as basis for destination grid. The default is the same as for GPW population data:
(-180.0, 89.99999999999991)
resampling (resampling function, optional) – The default is rasterio.warp.Resampling.bilinear
conserve (str, optional, either ‘mean’ or ‘sum’) – Conserve mean or sum of data? The default is None (no conservation).
- Returns
data_array_list (list) – contains reprojected data sets
meta_out (dict) – contains meta data of new grid (same for all arrays)
- climada.entity.exposures.litpop.litpop.gridpoints_core_calc(data_arrays, offsets=None, exponents=None, total_val_rescale=None)[source]¶
Combines N dense numerical arrays by point-wise multipilcation and optionally rescales to new total value: (1) An offset (1 number per array) is added to all elements in
the corresponding data array in data_arrays (optional).
Numbers in each array are taken to the power of the corresponding exponent (optional).
Arrays are multiplied element-wise.
if total_val_rescale is provided, results are normalized and re-scaled with total_val_rescale.
One array with results is returned.
- Parameters
data_arrays (list or array of numpy arrays containing numbers) – Data to be combined, i.e. list containing N (min. 1) arrays of same shape.
total_val_rescale (float or int, optional) – Total value for optional rescaling of resulting array. All values in result_array are skaled so that the sum is equal to total_val_rescale. The default (None) implies no rescaling.
offsets (list or array containing N numbers >= 0, optional) – One numerical offset per array that is added (sum) to the corresponding array in data_arrays. The default (None) corresponds to np.zeros(N).
exponents (list or array containing N numbers >= 0, optional) – One exponent per array used as power for the corresponding array. The default (None) corresponds to np.ones(N).
- Raises
ValueError – If input lists don’t have the same number of elements. Or: If arrays in data_arrays do not have the same shape.
- Returns
result_array – Results from calculation described above.
- Return type
np.array of same shape as arrays in data_arrays
climada.entity.exposures.litpop.nightlight module¶
- climada.entity.exposures.litpop.nightlight.NOAA_RESOLUTION_DEG = 0.008333333333333333¶
NOAA nightlights coordinates resolution in degrees.
- climada.entity.exposures.litpop.nightlight.NASA_RESOLUTION_DEG = 0.004166666666666667¶
NASA nightlights coordinates resolution in degrees.
- climada.entity.exposures.litpop.nightlight.NASA_TILE_SIZE = (21600, 21600)¶
NASA nightlights tile resolution.
- climada.entity.exposures.litpop.nightlight.NOAA_BORDER = (-180, -65, 180, 75)¶
NOAA nightlights border (min_lon, min_lat, max_lon, max_lat)
- climada.entity.exposures.litpop.nightlight.BM_FILENAMES = ['BlackMarble_%i_A1_geo_gray.tif', 'BlackMarble_%i_A2_geo_gray.tif', 'BlackMarble_%i_B1_geo_gray.tif', 'BlackMarble_%i_B2_geo_gray.tif', 'BlackMarble_%i_C1_geo_gray.tif', 'BlackMarble_%i_C2_geo_gray.tif', 'BlackMarble_%i_D1_geo_gray.tif', 'BlackMarble_%i_D2_geo_gray.tif']¶
Nightlight NASA files which generate the whole earth when put together.
- climada.entity.exposures.litpop.nightlight.load_nasa_nl_shape(geometry, year, data_dir=PosixPath('/home/docs/climada/data'), dtype='float32')[source]¶
Read nightlight data from NASA BlackMarble tiles cropped to given shape(s) and combine arrays from each tile. 1) check and download required blackmarble files 2) read and crop data from each file required in a bounding box around
the given geometry.
- combine data from all input files into one array. this array then
contains all data in the geographic bounding box around geometry.
return array with nightlight data
- Parameters
geometry (shape(s) to crop data to in degree lon/lat.) – for example shapely.geometry.(Multi)Polygon or shapefile.Shape. from polygon defined in a shapefile. The object should have attribute ‘bounds’ or ‘points’
year (int) – target year for nightlight data, e.g. 2016. Closest availble year is selected.
data_dir (Path (optional)) – Path to directory with BlackMarble data. The default is SYSTEM_DIR.
dtype (dtype) – data type for output default ‘float32’, required for LitPop, choose ‘int8’ for integer.
- Returns
results_array (numpy array) – extracted and combined nightlight data for bounding box around shape
meta (dict) – rasterio meta data for results_array
- climada.entity.exposures.litpop.nightlight.get_required_nl_files(bounds)[source]¶
- Determines which of the satellite pictures are necessary for
a certain bounding box (e.g. country)
- Parameters
bounds (1x4 tuple) – bounding box from shape (min_lon, min_lat, max_lon, max_lat).
- Raises
ValueError – invalid bounds
- Returns
req_files – Array indicating the required files for the current operation with a boolean value (1: file is required, 0: file is not required).
- Return type
numpy array
- climada.entity.exposures.litpop.nightlight.check_nl_local_file_exists(required_files=None, check_path=PosixPath('/home/docs/climada/data'), year=2016)[source]¶
Checks if BM Satellite files are avaialbe and returns a vector denoting the missing files.
- Parameters
required_files (numpy array, optional) – boolean array of dimension (8,) with which some files can be skipped. Only files with value 1 are checked, with value zero are skipped. The default is np.ones(len(BM_FILENAMES),)
check_path (str or Path) – absolute path where files are stored. Default: SYSTEM_DIR
year (int) – year of the image, e.g. 2016
- Returns
files_exist – Boolean array that denotes if the required files exist.
- Return type
numpy array
- climada.entity.exposures.litpop.nightlight.download_nl_files(req_files=array([1., 1., 1., 1., 1., 1., 1., 1.]), files_exist=array([0., 0., 0., 0., 0., 0., 0., 0.]), dwnl_path=PosixPath('/home/docs/climada/data'), year=2016)[source]¶
Attempts to download nightlight files from NASA webpage.
- Parameters
req_files (numpy array, optional) –
- Boolean array which indicates the files required (0-> skip, 1-> download).
The default is np.ones(len(BM_FILENAMES),).
files_exist (numpy array, optional) –
- Boolean array which indicates if the files already
exist locally and should not be downloaded (0-> download, 1-> skip). The default is np.zeros(len(BM_FILENAMES),).
dwnl_path (str or path, optional) – Download directory path. The default is SYSTEM_DIR.
year (int, optional) – Data year to be downloaded. The default is 2016.
- Raises
ValueError –
RuntimeError –
- Returns
dwnl_path – Download directory path.
- Return type
str or path
- climada.entity.exposures.litpop.nightlight.load_nasa_nl_shape_single_tile(geometry, path, layer=0)[source]¶
Read nightlight data from single NASA BlackMarble tile and crop to given shape.
- Parameters
geometry (shape or geometry object) – shape(s) to crop data to in degree lon/lat. for example shapely.geometry.Polygon object or from polygon defined in a shapefile.
path (Path or str) – full path to BlackMarble tif (including filename)
layer (int, optional) – TIFF-layer to be returned. The default is 0. BlackMarble usually comes with 3 layers.
- Returns
out_image[layer, (,:] : 2D numpy ndarray) – 2d array with data cropped to bounding box of shape
meta (dict) – rasterio meta
- climada.entity.exposures.litpop.nightlight.load_nightlight_nasa(bounds, req_files, year)[source]¶
Get nightlight from NASA repository that contain input boundary.
Note: Legacy for BlackMarble, not required for litpop module
- Parameters
bounds (tuple) – min_lon, min_lat, max_lon, max_lat
req_files (np.array) – array with flags for NASA files needed
year (int) – nightlight year
- Returns
nightlight (sparse.csr_matrix)
coord_nl (np.array)
- climada.entity.exposures.litpop.nightlight.read_bm_file(bm_path, filename)[source]¶
Reads a single NASA BlackMarble GeoTiff and returns the data. Run all required checks first.
Note: Legacy for BlackMarble, not required for litpop module
- Parameters
bm_path (str) – absolute path where files are stored.
filename (str) – filename of the file to be read.
- Returns
arr1 (array) – Raw BM data
curr_file (gdal GeoTiff File) – Additional info from which coordinates can be calculated.
- climada.entity.exposures.litpop.nightlight.unzip_tif_to_py(file_gz)[source]¶
Unzip image file, read it, flip the x axis, save values as pickle and remove tif.
- Parameters
file_gz (str) – file fith .gz format to unzip
- Returns
fname (str) – file_name of unzipped file
nightlight (sparse.csr_matrix)
- climada.entity.exposures.litpop.nightlight.untar_noaa_stable_nightlight(f_tar_ini)[source]¶
Move input tar file to SYSTEM_DIR and extract stable light file. Returns absolute path of stable light file in format tif.gz.
- Parameters
f_tar_ini (str) – absolute path of file
- Returns
f_tif_gz – path of stable light file
- Return type
str
- climada.entity.exposures.litpop.nightlight.load_nightlight_noaa(ref_year=2013, sat_name=None)[source]¶
Get nightlight luminosites. Nightlight matrix, lat and lon ordered such that nightlight[1][0] corresponds to lat[1], lon[0] point (the image has been flipped).
- Parameters
ref_year (int, optional) – reference year. The default is 2013.
sat_name (str, optional) – satellite provider (e.g. ‘F10’, ‘F18’, …)
- Returns
nightlight (sparse.csr_matrix)
coord_nl (np.array)
fn_light (str)