rhdx: a R package to interact with HDX

Ahmadou Dicko

6 minutes read


rhdx is an R package to interact with the HDX API.

It provide a series of utilities to facilitates interaction & analysis.

The Humanitarian Data Exchange (HDX) is an open platform for sharing data across crises and organisations. Launched in July 2014, the goal of HDX is to make humanitarian data easy to find and use for analysis. Our growing collection of datasets has been accessed by users in over 200 countries and territories.

Install rhdx

remotes::install_gitlab("dickoa/rhdx")
remotes::install_github("dickoa/rhdx")
remotes::install_gitlab("dickoa/rhdx")

Connect to the HDX server

library(rhdx)
set_rhdx_config(hdx_site = "prod")

You can check your config

get_rhdx_config()
## <HDX Configuration> 
##   HDX site: prod
##   HDX site url: https://data.humdata.org/
##   HDX API key:

Search datasets

List datasets matching a pattern

library(tidyverse)
search_datasets("ACLED Mali", rows = 2)
## [[1]]
## <HDX Dataset> 5895de63-010c-4716-97cb-fbdd3caf4e3a 
##   Title: Mali - Conflict Data
##   Name: acled-data-for-mali
##   Date: 01/01/1997-12/31/2019
##   Tags (up to 5): hxl, protests, security incidents, violence and conflict
##   Locations (up to 5): mli
##   Resources (up to 5): Conflict Data for Mali
## 
## attr(,"class")
## [1] "datasets_list"

Select the dataset

We will select the first dataset

search_datasets("ACLED Mali", rows = 2) %>%
  nth(1)
## <HDX Dataset> 5895de63-010c-4716-97cb-fbdd3caf4e3a 
##   Title: Mali - Conflict Data
##   Name: acled-data-for-mali
##   Date: 01/01/1997-12/31/2019
##   Tags (up to 5): hxl, protests, security incidents, violence and conflict
##   Locations (up to 5): mli
##   Resources (up to 5): Conflict Data for Mali

List dataset resources

search_datasets("ACLED Mali", rows = 2) %>%
  nth(1) %>%
  get_resources()
## [[1]]
## <HDX Resource> 54b56e5d-571b-4416-a1fa-4000fae6d15b 
##   Name: Conflict Data for Mali
##   Description: Conflict data with HXL tags
##   Size: 
##   Format: CSV

We select the first resource and read it directly in R

library(rhxl)
search_datasets("ACLED Mali", rows = 2) %>%
  nth(1) %>%
  get_resource(1) %>%
  read_resource(filename = "acled_mali.csv", hxl = TRUE)
## # A tibble: 3,221 x 31
##    data_id   iso event_id_cnty event_id_no_cnty event_date  year time_precision
##      <dbl> <dbl> <chr>                    <dbl> <date>     <dbl>          <dbl>
##  1 6404732   466 MLI3331                   3331 2019-12-07  2019              1
##  2 6405002   466 MLI3330                   3330 2019-12-07  2019              1
##  3 6404731   466 MLI3329                   3329 2019-12-06  2019              1
##  4 6404942   466 MLI3328                   3328 2019-12-05  2019              1
##  5 6404990   466 MLI3326                   3326 2019-12-05  2019              1
##  6 6404816   466 MLI3327                   3327 2019-12-05  2019              1
##  7 6404729   466 MLI3324                   3324 2019-12-04  2019              1
##  8 6404730   466 MLI3325                   3325 2019-12-04  2019              1
##  9 6404771   466 MLI3323                   3323 2019-12-03  2019              1
## 10 6404726   466 MLI3320                   3320 2019-12-02  2019              1
## # … with 3,211 more rows, and 24 more variables: event_type <chr>,
## #   sub_event_type <chr>, actor1 <chr>, assoc_actor_1 <chr>, inter1 <dbl>,
## #   actor2 <chr>, assoc_actor_2 <chr>, inter2 <dbl>, interaction <dbl>,
## #   region <chr>, country <chr>, admin1 <chr>, admin2 <chr>, admin3 <chr>,
## #   location <chr>, latitude <dbl>, longitude <dbl>, geo_precision <dbl>,
## #   source <chr>, source_scale <chr>, notes <chr>, fatalities <dbl>,
## #   timestamp <dbl>, iso3 <chr>

Save it for further use

search_datasets("ACLED Mali", rows = 2) %>%
  nth(1) %>%
  get_resource(1) %>%
  read_resource(filename = "acled_mali.csv", hxl = TRUE) -> acled_mali

Now let’s see where we have these conflicts in Mali

First step: just select the coordinates

acled_mali %>%
  select(longitude, latitude) %>%
  distinct() -> acled_mali_coord
acled_mali_coord
## # A tibble: 745 x 2
##    longitude latitude
##        <dbl>    <dbl>
##  1     1.46      15.8
##  2    -2.22      15.1
##  3    -3.75      14.2
##  4    -3.31      14.4
##  5     0.502     15.7
##  6    -3.78      14.1
##  7     0.696     15.0
##  8    -0.349     17.0
##  9     0.717     15.3
## 10    -2.57      14.3
## # … with 735 more rows

Second step download the official Mali COD boundaries from HDX

pull_dataset("administrative-boundaries-cod-mli") %>%
  get_resources()
## [[1]]
## <HDX Resource> 5a535eaf-25d6-4e6f-971a-2cb718aa0a97 
##   Name: MLI COD-AB 2019_08_07.pdf
##   Description: Mali COD-AB data sheet
##   Size: 102888
##   Format: PDF
## 
## [[2]]
## <HDX Resource> 9dcd9787-f2bb-4121-9003-d2267ce73895 
##   Name: MLI_AdminBoundaries_TabularData.xlsx
##   Description: Mali administrative level 0 (country), 1 (region or capital district), 2 (cercle), or 3 (commune) gazetteer
##   Size: 154595
##   Format: XLSX
## 
## [[3]]
## <HDX Resource> 186a227b-6e9c-4b90-8634-b6369da07e17 
##   Name: mli_adm_1m_dnct_2019_SHP.zip
##   Description: Mali administrative level 0 (country), 1 (region or capital district), 2 (cercle), or 3 (commune) shapefiles
##   Size: 2480801
##   Format: zipped shapefile
## 
## [[4]]
## <HDX Resource> d3fa6359-4c1b-455e-a753-250d8df603f2 
##   Name: mli_adm_1m_dnct_2019_EMF.zip
##   Description: Mali administrative level 0 (country), 1 (region or capital district), 2 (cercle), or 3 (commune) EMF files
##   Size: 2052742
##   Format: EMF
## 
## [[5]]
## <HDX Resource> 957e751b-8d5f-4755-ba68-c8dd614722f1 
##   Name: mli_adm_1m_dnct_2019_KMZ.zip
##   Description: Mali administrative level 0 (country), 1 (region or capital district), 2 (cercle), or 3 (commune) KMZ files
##   Size: 1270822
##   Format: ZIP
## 
## [[6]]
## <HDX Resource> 6daca6b4-6c35-4599-8126-d1851ace73fb 
##   Name: mli_adm_1m_dnct_2019.gdb.zip
##   Description: Mali administrative level 0 (country), 1 (region or capital district), 2 (cercle), or 3 (commune) geodatabase
##   Size: 2382299
##   Format: zipped geodatabase
## 
## [[7]]
## <HDX Resource> 89788941-205e-465e-bf1f-5c3b7422a71b 
##   Name: MLI_FR (MapServer)
##   Description: This map service contains OCHA Common Operational Datasets for Mali in French: Administrative Boundaries. The service is available as ESRI Map, ESRI Feature, WMS, and KML Services. See the OCHA COD/FOD terms of use for access and use constraints. 
##   Size: 
##   Format: LIVE SERVICE
## 
## [[8]]
## <HDX Resource> c65c3cf7-e4d5-4069-8f26-54ca6f49a22e 
##   Name: MLI_pcode (FeatureServer)
##   Description: This service is intended as a labelling layer for PCODES from OCHA's Common Operational Datasets for Mali. As a map service it is intended to be used in conjunction with the basemap located at http://gistmaps.itos.uga.edu/arcgis/rest/services/COD_External/MLI_FR/MapServer. The service also provides feature access to Administrative Boundaries geometry and taxonomy. The service is available as ESRI Map, WMS, ESRI Feature, WFS and KML Services. 
##   Size: 
##   Format: LIVE SERVICE
## 
## [[9]]
## <HDX Resource> b4ff6a29-c1c9-491c-8391-858281d644e7 
##   Name: MLI_pcode (MapServer)
##   Description: his service is intended as a labelling layer for PCODES from OCHA's Common Operational Datasets for Mali. As a map service it is intended to be used in conjunction with the basemap located at http://gistmaps.itos.uga.edu/arcgis/rest/services/COD_External/MLI_FR/MapServer. The service also provides feature access to Administrative Boundaries geometry and taxonomy. The service is available as ESRI Map, WMS, ESRI Feature, WFS and KML Services. 
##   Size: 
##   Format: LIVE SERVICE

We just need the country boundary (admin 0) which is the fourth resource

pull_dataset("administrative-boundaries-cod-mli") %>%
  get_resource(4) %>%
  read_resource() -> adm0_mali

ggplot2 is a R package for graphics with a nice grammar to compose any type of visual, we also use sf which add spatial data support.

library(sf)
ggplot() +
  geom_sf(data = adm0_mali) +
  geom_point(data = acled_mali_coord, aes(x = longitude, y = latitude)) +
  theme_bw()

comments powered by Disqus