BlackMarble class

This class is used to approximate the economic exposure. It models countries and provinces interpolating the GDP and income group values through the night light intensities for a specific year.

The night light images used are the following: - year 2012 and 2016: https://earthobservatory.nasa.gov/Features/NightLights (15 arcsec resolution (~500m)) - from 1992 until 2013: https://ngdc.noaa.gov/eog/dmsp/downloadV4composites.html (30 arcsec resolution (~1km), stable lights)

By default, for years higher than 2013 the NASA images are used, whilst for 2013 and earlier years the NOAA ones are considered. However, there is a flag which allows to choose the closest NASA or NOAA images. The default resolution is that of the image, but any resolution can be set and will be computed by interpolation.

Regarding the GDP (nominal GDP at current USD) and income group values, they are obtained from the World Bank using the pandas-datareader API. If a value is missing, the value of the closest year is considered. When no values are provided from the World Bank, we use the Natural Earth repository values.

The BlackMarble class inherits from the `Exposures <climada_entity_Exposures.ipynb#Exposures-class>`__ class. It provides a set_countries() method which enables to model a country using different settings. The first time a night light image is used, it is downloaded and stored locally. This might take some time.

With the next commands, we can model Iran and Turkey in the years 1992 and 2013 with the default resolution of ~1km. Please take into account that the NOAA images need to be calibrated in order to perform proper comparision between years.

[1]:

%matplotlib inline
from climada.entity import BlackMarble

irn_tur_1992 = BlackMarble()
irn_tur_1992.set_countries(['Iran', 'Turkey'], 1992, res_km=1.0)
irn_tur_1992.plot_hexbin(vmax=1e7)

irn_tur_2013 = BlackMarble()
irn_tur_2013.set_countries(['Iran', 'Turkey'], 2013, res_km=1.0)
irn_tur_2013.plot_hexbin(vmax=1e7)

2019-05-14 18:24:01,471 - climada - DEBUG - Loading default config file: /Users/aznarsig/Documents/Python/climada_python/climada/conf/defaults.conf
2019-05-14 18:24:04,214 - climada.util.finance - INFO - GDP IRN 1993: 6.374e+10.
2019-05-14 18:24:04,287 - climada.util.finance - INFO - Income group IRN 1992: 2.
2019-05-14 18:24:05,127 - climada.util.finance - INFO - GDP TUR 1992: 1.585e+11.
2019-05-14 18:24:05,193 - climada.util.finance - INFO - Income group TUR 1992: 2.
2019-05-14 18:24:05,193 - climada.entity.exposures.black_marble - INFO - Nightlights from NOAA's earth observation group for year 1992.
2019-05-14 18:24:05,330 - climada.entity.exposures.black_marble - INFO - Processing country Iran.
2019-05-14 18:24:09,112 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.
2019-05-14 18:24:09,327 - climada.entity.exposures.black_marble - INFO - Processing country Turkey.
2019-05-14 18:24:11,130 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '

2019-05-14 18:24:33,430 - climada.util.finance - INFO - GDP IRN 2013: 4.674e+11.
2019-05-14 18:24:33,497 - climada.util.finance - INFO - Income group IRN 2013: 3.
2019-05-14 18:24:34,620 - climada.util.finance - INFO - GDP TUR 2013: 9.506e+11.
2019-05-14 18:24:34,671 - climada.util.finance - INFO - Income group TUR 2013: 3.
2019-05-14 18:24:34,672 - climada.entity.exposures.black_marble - INFO - Nightlights from NOAA's earth observation group for year 2013.
2019-05-14 18:24:34,866 - climada.entity.exposures.black_marble - INFO - Processing country Iran.
2019-05-14 18:24:38,570 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.
2019-05-14 18:24:38,831 - climada.entity.exposures.black_marble - INFO - Processing country Turkey.
2019-05-14 18:24:40,582 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '

[1]:

(<Figure size 648x936 with 2 Axes>,
 array([[<cartopy.mpl.geoaxes.GeoAxesSubplot object at 0x1a25a1cd30>]],
       dtype=object))
../_images/tutorial_climada_entity_BlackMarble_2_5.png
../_images/tutorial_climada_entity_BlackMarble_2_6.png 
[2]:

irn_tur_1992.value.size

[2]:

3417059

Higher resolution is obtained by interpolating the night lights. If the resolution (parameter res_km) is not provided, the resolution of the nightlight images is going to be used.

[3]:

che = BlackMarble()
che.set_countries(['Switzerland'], 2013, res_km=0.3)
che.plot_hexbin()

# use the NASA files (only available for 2012 and 2016) to interpolate from higher resolution
che = BlackMarble()
che.set_countries(['Switzerland'], 2013, res_km=0.3, from_hr=True)
che.plot_hexbin()

2019-05-14 18:26:01,867 - climada.util.finance - INFO - GDP CHE 2013: 6.885e+11.
2019-05-14 18:26:01,934 - climada.util.finance - INFO - Income group CHE 2013: 4.
2019-05-14 18:26:01,935 - climada.entity.exposures.black_marble - INFO - Nightlights from NOAA's earth observation group for year 2013.
2019-05-14 18:26:02,172 - climada.entity.exposures.black_marble - INFO - Processing country Switzerland.
2019-05-14 18:26:02,293 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 0.3 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '

2019-05-14 18:26:15,078 - climada.util.finance - INFO - GDP CHE 2013: 6.885e+11.
2019-05-14 18:26:15,128 - climada.util.finance - INFO - Income group CHE 2013: 4.
2019-05-14 18:26:15,129 - climada.entity.exposures.black_marble - INFO - Nightlights from NASA's earth observatory for year 2012.
2019-05-14 18:26:15,130 - climada.entity.exposures.nightlight - DEBUG - Found all required satellite data (1 files) in folder /Users/aznarsig/Documents/Python/climada_python/data/system
2019-05-14 18:26:15,131 - climada.entity.exposures.nightlight - DEBUG - All required files already exist. No downloads neccessary.
2019-05-14 18:26:25,857 - climada.entity.exposures.black_marble - INFO - Processing country Switzerland.
2019-05-14 18:26:26,403 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 0.3 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '

[3]:

(<Figure size 648x936 with 2 Axes>,
 array([[<cartopy.mpl.geoaxes.GeoAxesSubplot object at 0x1a258c0f28>]],
       dtype=object))
../_images/tutorial_climada_entity_BlackMarble_5_5.png
../_images/tutorial_climada_entity_BlackMarble_5_6.png 
[4]:

chn = BlackMarble()
chn.set_countries(['China'], 2013, res_km=5.0)
chn.plot_hexbin(vmax=5e8)

2019-05-14 18:27:09,391 - climada.util.finance - INFO - GDP CHN 2013: 9.607e+12.
2019-05-14 18:27:09,457 - climada.util.finance - INFO - Income group CHN 2013: 3.
2019-05-14 18:27:09,458 - climada.entity.exposures.black_marble - INFO - Nightlights from NOAA's earth observation group for year 2013.
2019-05-14 18:27:09,628 - climada.entity.exposures.black_marble - INFO - Processing country China.
2019-05-14 18:27:39,011 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 5.0 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '

[4]:

(<Figure size 648x936 with 2 Axes>,
 array([[<cartopy.mpl.geoaxes.GeoAxesSubplot object at 0x1a258ac048>]],
       dtype=object))
../_images/tutorial_climada_entity_BlackMarble_6_3.png

Sea points can be added with the desired resolution. Their value will be 0.

[5]:

aus = BlackMarble()
aus.set_countries(['Australia'], 2013, res_km=2.0, sea_res=(100, 10))
from matplotlib import colors
norm=colors.LogNorm(vmin=1.0e4, vmax=1.0e9)
aus.plot_hexbin(norm=norm)

2019-05-14 18:28:01,370 - climada.util.finance - INFO - GDP AUS 2013: 1.574e+12.
2019-05-14 18:28:01,418 - climada.util.finance - INFO - Income group AUS 2013: 4.
2019-05-14 18:28:01,418 - climada.entity.exposures.black_marble - INFO - Nightlights from NOAA's earth observation group for year 2013.
2019-05-14 18:28:01,575 - climada.entity.exposures.black_marble - INFO - Processing country Australia.
2019-05-14 18:28:20,673 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 2.0 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '

[5]:

(<Figure size 648x936 with 2 Axes>,
 array([[<cartopy.mpl.geoaxes.GeoAxesSubplot object at 0x1a25be0cc0>]],
       dtype=object))
../_images/tutorial_climada_entity_BlackMarble_8_3.png

If you don’t write the country name right, the error log will provide you with possible similar options.

[6]:

hkg = BlackMarble()
try:
    hkg.set_countries(['Hong Kong'], 2016, res_km=0.2)
except ValueError:
    print('Error caught.')
    hkg.set_countries(['Hong Kong S.A.R.'], 2016, res_km=0.2)
    hkg.plot_hexbin(gridsize=500)

2019-05-14 18:29:17,462 - climada.entity.exposures.black_marble - ERROR - Country Hong Kong not found. Possible options: ['Hong Kong S.A.R.']
Error caught.
2019-05-14 18:29:18,409 - climada.util.finance - INFO - GDP HKG 2016: 3.209e+11.
2019-05-14 18:29:18,465 - climada.util.finance - INFO - Income group HKG 2016: 4.
2019-05-14 18:29:18,465 - climada.entity.exposures.black_marble - INFO - Nightlights from NASA's earth observatory for year 2016.
2019-05-14 18:29:18,469 - climada.entity.exposures.nightlight - DEBUG - Found all required satellite data (1 files) in folder /Users/aznarsig/Documents/Python/climada_python/data/system
2019-05-14 18:29:18,469 - climada.entity.exposures.nightlight - DEBUG - All required files already exist. No downloads neccessary.
2019-05-14 18:29:25,919 - climada.entity.exposures.black_marble - INFO - Processing country Hong Kong S.A.R..
2019-05-14 18:29:25,944 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 0.2 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/scipy/ndimage/interpolation.py:605: UserWarning: From scipy 0.13.0, the output shape of zoom() is calculated with round() instead of int() - for these inputs the size of the returned array has changed.
  "the returned array has changed.", UserWarning)
/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '
../_images/tutorial_climada_entity_BlackMarble_10_2.png

The GDP and income group can be given as inputs. Void values will indicate that the ones computed by Black Marble are used.

[7]:

au = BlackMarble()
au.set_countries(['Argentina', 'Uruguay'], 2013, res_km=1.0, gdp={'ARG': 5e11, 'URY': 5e10}, inc_grp={'ARG': 3, 'URY': 4})
norm=colors.LogNorm(vmin=1.0e5, vmax=1.0e8)
au.plot_hexbin(pop_name=False, norm=norm)

2019-05-14 18:29:28,233 - climada.entity.exposures.black_marble - INFO - Nightlights from NOAA's earth observation group for year 2013.
2019-05-14 18:29:28,390 - climada.entity.exposures.black_marble - INFO - Processing country Argentina.
2019-05-14 18:29:36,547 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.
2019-05-14 18:29:36,984 - climada.entity.exposures.black_marble - INFO - Processing country Uruguay.
2019-05-14 18:29:37,327 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.

[7]:

(<Figure size 648x936 with 2 Axes>,
 array([[<cartopy.mpl.geoaxes.GeoAxesSubplot object at 0x1a3d128be0>]],
       dtype=object))
../_images/tutorial_climada_entity_BlackMarble_12_2.png

Finally, also provinces can be modelled using the country data as follows:

[13]:

country_name = {'Germany': ['Berlin', 'Brandenburg', 'Bayern', 'Sachsen', 'Thüringen', 'Sachsen-Anhalt'],
                'Czechia': ['Prague', 'Karlovarský', 'Ã\x9astecký', 'Liberecký', 'StÅ\x99edoÄ\x8deský', 'PlzeÅ\x88ský', 'JihoÄ\x8deský']}

ent = BlackMarble()
ent.set_countries(country_name, 2012, res_km=1.0, from_hr=True)

from matplotlib import colors
norm=colors.LogNorm(vmin=1.0e6, vmax=5.0e9)
ent.plot_hexbin(norm=norm)

2019-05-14 19:04:12,687 - climada.util.finance - INFO - GDP DEU 2012: 3.544e+12.
2019-05-14 19:04:12,743 - climada.util.finance - INFO - Income group DEU 2012: 4.
2019-05-14 19:04:13,909 - climada.util.finance - INFO - GDP CZE 2012: 2.074e+11.
2019-05-14 19:04:13,960 - climada.util.finance - INFO - Income group CZE 2012: 4.
2019-05-14 19:04:13,961 - climada.entity.exposures.black_marble - INFO - Nightlights from NASA's earth observatory for year 2012.
2019-05-14 19:04:14,003 - climada.entity.exposures.nightlight - DEBUG - Found all required satellite data (1 files) in folder /Users/aznarsig/Documents/Python/climada_python/data/system
2019-05-14 19:04:14,004 - climada.entity.exposures.nightlight - DEBUG - All required files already exist. No downloads neccessary.
2019-05-14 19:04:24,700 - climada.entity.exposures.black_marble - INFO - Processing country Germany.
2019-05-14 19:04:28,702 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.
2019-05-14 19:04:29,348 - climada.entity.exposures.black_marble - INFO - Processing country Czechia.
2019-05-14 19:04:30,289 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '

[13]:

(<Figure size 648x936 with 2 Axes>,
 array([[<cartopy.mpl.geoaxes.GeoAxesSubplot object at 0x1a25898470>]],
       dtype=object))
../_images/tutorial_climada_entity_BlackMarble_14_3.png

Countries with oversea territories might take longer to compute. The USA using 0.5km resolution might take longer… (10min approx)

[14]:

country_name = {'United States Of America': ['Florida']}

flr = BlackMarble()
flr.set_countries(country_name, 2017)
flr.plot_hexbin()
print('Number of points:', flr.size)

2019-05-14 19:04:46,241 - climada.util.finance - INFO - GDP USA 2017: 1.949e+13.
2019-05-14 19:04:46,298 - climada.util.finance - INFO - Income group USA 2017: 4.
2019-05-14 19:04:46,299 - climada.entity.exposures.black_marble - INFO - Nightlights from NASA's earth observatory for year 2016.
2019-05-14 19:04:46,981 - climada.entity.exposures.nightlight - DEBUG - Found all required satellite data (4 files) in folder /Users/aznarsig/Documents/Python/climada_python/data/system
2019-05-14 19:04:46,981 - climada.entity.exposures.nightlight - DEBUG - All required files already exist. No downloads neccessary.
2019-05-14 19:09:08,067 - climada.entity.exposures.black_marble - INFO - Processing country United States Of America.
2019-05-14 19:15:44,820 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 0.5 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '

Number of points: 4287405
../_images/tutorial_climada_entity_BlackMarble_16_3.png

The polynomial transformation used in the nightlight intensity can be also set through the parameter poly_val. The default transformation is x^2; not transforming the intensities has this result:

[12]:

country_name = {'Germany': ['Berlin', 'Brandenburg', 'Bayern', 'Sachsen', 'Thüringen', 'Sachsen-Anhalt'],
                'Czechia': ['Prague', 'Karlovarský', 'Ã\x9astecký', 'Liberecký', 'StÅ\x99edoÄ\x8deský', 'PlzeÅ\x88ský', 'JihoÄ\x8deský']}

ent = BlackMarble()
# linear transformation
ent.set_countries(country_name, 2012, res_km=1.0, from_hr=True, poly_val=[0, 1])

from matplotlib import colors
norm=colors.LogNorm(vmin=1.0e6, vmax=5.0e9)
ent.plot_hexbin(norm=norm)

2019-05-14 19:03:25,682 - climada.util.finance - INFO - GDP DEU 2012: 3.544e+12.
2019-05-14 19:03:25,735 - climada.util.finance - INFO - Income group DEU 2012: 4.
2019-05-14 19:03:26,595 - climada.util.finance - INFO - GDP CZE 2012: 2.074e+11.
2019-05-14 19:03:26,644 - climada.util.finance - INFO - Income group CZE 2012: 4.
2019-05-14 19:03:26,645 - climada.entity.exposures.black_marble - INFO - Nightlights from NASA's earth observatory for year 2012.
2019-05-14 19:03:26,674 - climada.entity.exposures.nightlight - DEBUG - Found all required satellite data (1 files) in folder /Users/aznarsig/Documents/Python/climada_python/data/system
2019-05-14 19:03:26,675 - climada.entity.exposures.nightlight - DEBUG - All required files already exist. No downloads neccessary.
2019-05-14 19:03:37,715 - climada.entity.exposures.black_marble - INFO - Processing country Germany.
2019-05-14 19:03:41,721 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.
2019-05-14 19:03:42,360 - climada.entity.exposures.black_marble - INFO - Processing country Czechia.
2019-05-14 19:03:43,467 - climada.entity.exposures.black_marble - INFO - Generating resolution of approx 1.0 km.

/Users/aznarsig/anaconda3/envs/climada_up/lib/python3.7/site-packages/matplotlib/tight_layout.py:176: UserWarning: Tight layout not applied. The left and right margins cannot be made large enough to accommodate all axes decorations.
  warnings.warn('Tight layout not applied. The left and right margins '

[12]:

(<Figure size 648x936 with 2 Axes>,
 array([[<cartopy.mpl.geoaxes.GeoAxesSubplot object at 0x1a3218cc50>]],
       dtype=object))
../_images/tutorial_climada_entity_BlackMarble_18_3.png 
[ ]: