CALCULUS | Marcin Stępniak

CALCULUS project main results

Sun, 09 Sep 2018 00:00:00 +0000

This section contains main project results. They are divided into three main parts:

Temporal resolution: The study aims to to provide guidance to researchers and practitioners on the selection of appropriate temporal resolutions in accessibility studies.
GTFS study: This research focus on an evaluation of a supply side of public transport using GTFS and other data available in open access.
Policy support tool: This section contains a brief summary of methodological guidelines for an evaluation of an impact of particular transport related restrictions of accessibility in urban area. The method extensively uses a new, time-sensitive network data (speed profiles and GTFS feeds) and it is based on comparison of accessibility levels and its spatial patterns in order to identify a particular, transport-related accessibility restrictions.

{DepartureTime} R package

Mon, 18 May 2020 00:00:00 +0000

I have created my first (ever!) #rstats package. It is called {DepartureTime} and its purpose is to prepare a dataset with several departure times for temporally sensitive accessibility analysis.

What does `{DepartureTime}` do?

The package consists of one function which permits you to generate a series of departure times applying user-defined temporal resolution and one of four different sampling procedures:

Systematic sampling method: departure times are selected using a regular interval defined by the frequency
Simple Random sampling method: a specified number of departure times (defined by the frequency) is randomly selected from the time window
Hybrid sampling method: departure times are randomly selected from given time intervals (resulted from applied temporal resolution)
Constrained Random Walk Sampling sampling method: a first departure time is randomly selected from the subset of the length defined by the frequency and beginning of the time window; then, the next departure time is randomly selected from the subset limited by \(Tn+f/2\) and \(Tn+f+f/2\).

An example of the result of sampling procedures for 20-minute temporal resolution and 1-hour-long time window (07:00-08:00) illustrates the following table:

Sampling method	Departure times	Comments
Systematic	07:00; 07:20, 7:40, 08:00	regular interval of 20 minutes¹
Simple Random	07:18; 07:51; 07:55	3 randomly selected departure times from the time window²
Hybrid	07:02; 07:23; 07:50	One randomly selected departure time from each time interval period³
Random Walk	07:15; 07:36; 07:49	on average there should be 20-minute interval between departure times⁴

¹ as 20-minute interval fits to 60 minute time window it provides 4 departure times.
² i.e. one per each 20 min. in 60-minute time window.
³ i.e. one from 07:00-07:19, one from 07:20-07:39 and one from 07:40-07:59.
⁴ due to the nature of the sampling procedure, the number of departure times might differ.

For details on sampling procedures, please consult Owen & Murphy (2018).

Why may you need `{DepartureTime}`?

Briefly: because, if you include temporal dimension to your accessibility analysis, the result depends on the applied departure time. In particular, in case of public transport, accessibility level in a given area might be highly affected by the fact whether you are “lucky” or not in terms of waiting times (for the first connection and/or in case of transfers) and, in result, these values might differ substantially just because you decided on a particular departure time. In order to limit this negative impact on analysis, you need to consider different departure times and than aggregate results in order to better reflect how does one experience level of accessibility.

Consider this graph presented by Owen & Levinson (2015):

{width=400px}

It shows how accessibility change over time for a selected census block. Depending on the selected departure time, you can get completely different accessibility level, even though the transport system nor the distribution of activities does not change.

Further, even if the level of availability does not change so drastically, you may still want to simulate different situation e.g. applying free flow, peak or out-of-peak speeds. The graph below, prepared for the Dutch case study, compares daily variation of job accessibility by car and by public transport (walk-and-ride and bike-and-ride models):

Thus, regardless transport mode, you may need to repeat analysis without changing anything but departure time (even though, a required temporal resolution would change): in case of car accessibility you may need to generate origin-destination matrices couple of times during the day, while in case of public transport you should compute travel times even couple of times per hour (I you need more information on the consequences of applied temporal resolution on precision of accessibility analysis I can shamelessly suggest you the following paper: Stepniak, Pritchard, Geurs & Goliszek (2019)).

How can you take advantage of `{DepartureTime}`?

As described in the paper, you can select different approach how to tackle the issue of temporal resolution. Regardless which approach you select, you would need a table which contains all departure times in order to automatize calculations. I used it in ArcGIS Network Analyst (with Add GTFS to a Network Dataset tool), but as far as I know, {DepartureTime} may be also useful when working with OpenTripPlanner.

In ArcGIS you can easily prepare an arcpy code or just a ModelBuilder, which permits you to iterate by subsequent departure time. The simple ModelBuilder looks like this:

The .dbf file with the output from {DepartureTime} you need to locate in Departure Times (market with the red circle) and iterate it by Field Values (by Date field, setting Data type as Date)

In result of the above model, you obtain a set of origin-destination matrices, one for each of a departure time, exported to separate files (e.g. .dbf files). Then, they can be aggregated in order to obtain more realistic average travel time (or average accessibility), e.g. during the morning peak-hours or during the day (night etc.).

How does `{DepartureTime}` work?

The {DepartureTime} package can be installed in R directly from GitHub (if you don’t have {devtools} installed, you need to install it first):

# install.packages("devtools")
devtools::install_github("stmarcin/DepartureTime")

The function has the following syntax and default values:

DepartureTime <- function(method = "H",
                          dy = format(Sys.Date(), "%Y"),  
                          dm = format(Sys.Date(), "%m"), 
                          dd = format(Sys.Date(), "%d"),
                          tmin = 0, tmax = 24,
                          res = 5,
                          MMDD = TRUE,
                          ptw = FALSE)

Function variables:

method - sampling method; Options:
- R OR Random: Simple random sampling;
- S OR Systematic: Systematic sampling;
- H OR Hybrid: Hybrid sampling;
- W OR Walk: Constrained random walk sampling;
dy, dm and dd - date of the analysis (formats: YYYY, MM, DD); default: system date;
tmin and tmax - limits of the time window (format: HH); default: full day (00:00 - 24:00);
res - temporal resolution; default: 5 minutes
MMDD - date format of the output (TRUE / FALSE) default: TRUE
- TRUE: MM/DD/YYYY;
- FALSE: DD/MM/YYYY;
ptw - print limits of subsetted time-windows; default: FALSE;

The DepartureTime() function creates a data frame with generated departure times, already formatted for ArcGIS:

ColumnName	Description
ID	rowID (integer), starts with 0
Date	Departure date & hour

Example with selected user-defined parameters:

DepartureTime(method = "S",    # systematic sampling method
  dm = 5, dd = 15,             # user-defined date: 15th May, 2020 (current year)
  tmin = 7, tmax = 9,          # user-defined time window (07:00 - 09:00)
  res = 20)                    # user-defined temporal resolution (20 minutes)

ID	Date
0	05/15/2020 07:00
1	05/15/2020 07:20
2	05/15/2020 07:40
3	05/15/2020 08:00
4	05/15/2020 08:20
5	05/15/2020 08:40
6	05/15/2020 09:00

You can easily save {DepartureTime} output as .dbf file (you need to have {foreign} package installed):

library(DepartureTime)
library(foreign)
library(dplyr)

# generate departure times for 8-10am time window 
# with 30-minute temporal resolution applying hybrid sampling model:
DepartureTime(tmin = 8, tmax = 10, res = 30) %>% 
  
  #save output in OD_analysis subfolder as My_Departure_Times.dbf
  write.dbf("OD_analysis/My_Departure_Times.dbf")

Questions?

If you have any questions, feel free to contact me or fill an issue on github.

Interactive choropleth maps with R and tmap (part I)

Wed, 25 Sep 2019 00:00:00 +0000

In this post, I would like to share how to prepare an interactive map using #rstats and {tmap} package. The first part shows how to make a simple (one thematic layer), interactive choropleth map and save an output as a .html file, which can be then inserted to a website or serve as a stand alone web. A {tmap} provides with a certain interactivity: zoom in/out and map moving, a popup display and a selection of visibility of basemaps and/or thematic layers.

The map shows an total number of accessible jobs from a given location (transport zone), applying a typical commuting distance of inhabitants of Metropolitan area of Madrid (more details about used potential accessibility measure, can be found here).

Data used for this example can be downloaded from the github. The accessibility values were calculated for the MSCA CALCULUS project. The shapefiles of transport zones of Madrid can be downloaded from the open data portal of the Comunidad de Madrid (Delimitaciones Territoriales / Zonas de Transporte). In this map I use the zones dated for year 2013 limited to those located within the Municipality of Madrid (584 units).

Step 1. Load packages and get the data

library(readr)
library(dplyr)
library(tmap)
library(rgdal)

For the map we need a shapefile with transport zones of Madrid (Madrid_TAZ.shp) and csv file with accessibility values, both stored in the [Data] subfolder. The code below loads a shapefile (as SpatialPolygonsDataFrame) and merges it with the .csv. Additionally, the code recalculates the total number of accessible jobs (an original Ai_tmap.csv files contains results in thousands) which will be used in the popup.

Free.Flow <- merge(
      
      # read shapefile
      readOGR("Data/Madrid_TAZ.shp",
            layer = "Madrid_TAZ", GDAL1_integer64_policy = TRUE),
      
      # read csv file add a new column to be displayed in popup
      (read_csv("Data/Ai_tmap.csv") %>%
            mutate(K.FreeFlow = FreeFlow*1000) ),
      
      # matching columns
      by.x = "TAZ_Madr_1", by.y = "Or")

Step 2. Make a (t)map!

First, let’s set the tmap “view” mode:

tmap_mode("view")

The advantage of using tmap is that it shares a logic with ggplot. Thus, first we need to create a tmap object (tm_shape()) followed by a thematic layer (tm_polygons()), and then additional parameters (e.g. tm_layout()) should be specified. The subsequent elements we add using a +.

Let’s start with the first map using all default settings. We need to define the source of thematic layer (Free.Flow - a SpatialPolygonsDataFrame object) and a variable which defines a thematic layer (FreeFlow):

tm_shape(Free.Flow) +
    tm_polygons("FreeFlow")

Here it is! A first interactive map. In the top-left corner, we can turn off the visibility of our thematic layer and select one of the three default basemaps.

The popup shows us a value taken from the column used to create a thematic layer (FreeFlow) and it uses a first column to define area ID. We can change it now, using a popup.vars parameter of tm_polygons() function. The content of the popup is defined as: some.text = variable.name and a variable.name should be matched to the name of column of a thematic layer (as defined by tm_shape()). We use two columns: K.FreeFlow which contains a total number of available jobs, and POP2017 indicating a total number of population:

tm_shape(Free.Flow) +
    tm_polygons("FreeFlow",
                # popup definition
                popup.vars=c(
                    "Accessible_jobs: "="K.FreeFlow",
                    "Population: " = "POP2017")  
                )

One remark: by default the popup uses a first column of our SpatialPolygonsDataFrame to display an area ID. We can use variables from another column by defining an id parameter:

tm_shape(Free.Flow) +
      tm_polygons("FreeFlow",
                    
                    # popup definition
                    popup.vars=c(
                            "Accessible_jobs: "="K.FreeFlow",
                            "Population: " = "POP2017"),
                    id = "TAZ_Madrid"
        )

Now, let’s change the color palette and transparency of the thematic layer. I want to use the inferno palette - a viridis-based palette matched by {colorspace} team. I want to divide my data into as much as 16 classes. Additionally, I want to set the transparency of the thematic layer. In result, I need to define three parameters: palette, n (number of classes) and alpha (transparency):

        alpha = 0.6,
        n = 16,
        palette = hcl.colors(16, palette = "Inferno")

However, the darkest colors of the inferno palette are a bit too dark for me, so I want to exclude them. In order to do this, first I set more colors from the palette, and then filter out first two of them. Now, our code looks like this:

tm_shape(Free.Flow) +
        tm_polygons("FreeFlow",
                    
                    # popup definition
                    popup.vars=c(
                            "Accessible_jobs: "="K.FreeFlow",
                            "Population: " = "POP2017"),
                    id = "TAZ_Madrid",
                    
                    # transparency, number of classes and palette
                    alpha = 0.6,
                    n = 16,
                    palette = hcl.colors(18, palette = "Inferno")[3:18]
        )

The zones’ borders are too visible so we need to increase their transparency and change their color. I’ve tested several solutions taking an advantage of this on-line application and selected the one that fits the best:

tm_shape(Free.Flow) +
        tm_polygons("FreeFlow",
                    
                    # popup definition
                    popup.vars=c(
                            "Accessible_jobs: "="K.FreeFlow",
                            "Population: " = "POP2017"),
                    id = "TAZ_Madrid",
                    
                    # transparency, number of classes and palette
                    alpha = 0.6,
                    n = 16,
                    palette = hcl.colors(18, palette = "Inferno")[3:18],
                    
                    # border definition: color and transparency
                    border.col = "#990099",
                    border.alpha = 0.1
        )

The map is almost ready. The last amendments: title of the legend and the map title. The previous is defined by the title parameter of tm_polygons() function, while the latter by a parameter of a separate function tm_layout():

tm_shape(Free.Flow) +
        tm_polygons("FreeFlow",
                    
                    # popup definition
                    popup.vars=c(
                            "Accessible_jobs: "="K.FreeFlow",
                            "Population: " = "POP2017"),
                    id = "TAZ_Madrid",
                    
                    # transparency, number of classes and palette
                    alpha = 0.6,
                    n = 16,
                    palette = hcl.colors(18, palette = "Inferno")[3:18],
                    
                    # border definition: color and transparency
                    border.col = "#990099",
                    border.alpha = 0.1, 
                    
                    # title of the legend
                    title = "Accessible jobs<br>(thous.)"
        ) +
    
    # map title 
    tm_layout(title = "Accessibility to jobs<br>Model: Car free flow speeds")

Note, that in both cases, a typical html tags (like <br>) can be used.

Step 3. Save and re-use the output

The map is ready! Now, we only need to save it as a html widget. We can use a native tmap_save() function:

tmap_last() %>% 
    tmap_save("Madrid_accessibility_map.html")

It can be then inserted to a web page:

<iframe src="/img/Madrid_accessibility_map.html" frameborder=0,  height=400, width="100%", scrolling="no"></iframe>

All the code and the data can be downloaded from my github.

Session info:

R version 3.6.0 (2019-04-26)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.5

Some useful links:

tmap: get started!
tmap in a nutshell
The greate on-line course Using R for Data Journalism by Andrew Ba Tran
The chapter in Geocomputation with R by Robin Lovelace, Jakub Nowosad and Jannes Muenchow