current time as POSIXct:

unclass(Sys.time())

## [1] 1556570382

Sys.time()-1556568634

## [1] "1970-01-01 02:29:08 IST"

current time as POSIXit:

unclass(as.POSIXlt(Sys.time()))

## $sec
## [1] 42.50493
## 
## $min
## [1] 39
## 
## $hour
## [1] 23
## 
## $mday
## [1] 29
## 
## $mon
## [1] 3
## 
## $year
## [1] 119
## 
## $wday
## [1] 1
## 
## $yday
## [1] 118
## 
## $isdst
## [1] 1
## 
## $zone
## [1] "IDT"
## 
## $gmtoff
## [1] 10800
## 
## attr(,"tzone")
## [1] ""    "IST" "IDT"

• usually POSIXct is easier to work with… with lubridate you don’t need to worry about it

in this workshop we will give introduction to ‘lubridate’ package

Convert to Datetime object (class POSIXCT/POSIXIT)

lubridate has multiple functions that convert factor (or any other class) to dates. For example:

• ymd() (year, month, day)
• mdy()
• dmy()

The difference is the order of the time variables–you need to choose the one that fits for your date type.

e.g. in the date_1 column the date is built in the day-month-year format so I use the dmy function

timeSample %>% 
  mutate(date_1 = dmy(date_1)) %>% 
  kable()

## Warning: package 'bindrcpp' was built under R version 3.4.4

in date_2 column the date is built in format of year-month-day so i use the ymd function

timeSample %>% 
  mutate(date_1 = dmy(date_1),
         date_2 = ymd(date_2)) %>% 
  kable()

Note the only thing that matters is the order of the day/month/year. The separators can change (‘:’ or ‘/’ or ‘-’ etc.) and lubridate can handle it…the same goes for 02 or 2, 2019 or 19.

Really cool!!

lubridate can also handle date and time object in the same manner… * note - can translate AM/PM to the right 24 clock hour

timeSample <- timeSample %>% 
  mutate(date_1 = dmy(date_1),
         date_2 = ymd(date_2),
         datetime_1 = dmy_hms(datetime_1)) 

Time zones

When dealing with times in R it’s super important to pay attention to time zones

If you don’t specify the time zone, the default will be UTC. This can cause problems later on if you do all kind of time calculations and don’t understand why you get weird numbers (;

The force_tz function will take the time as it and change the time zone (in the ‘back office’) to the required one. It’s good for cases where you didn’t specify the time zone for your data beforehand and now you want to fix it.

tz(timeSample$datetime_1)

## [1] "UTC"

timeSample$datetime_1 <- force_tz(timeSample$datetime_1, "Asia/Jerusalem")
tz(timeSample$datetime_1)

## [1] "Asia/Jerusalem"

The with_tz function will convert your time zone to a different one. this function is good for cases where you want to know the time in different places

timeSample$datetime_utc <- with_tz(timeSample$datetime_1, "UTC")

The best practice is to specify the timezone in first place

ymd_hms("2019-07-20 08:35:00", tz = "Asia/Jerusalem")

## [1] "2019-07-20 08:35:00 IDT"

Other nice functions in lubridate

days_in_month(today()) # how many days in this month?

## Apr 
##  30

days_in_month(today() + 31) # how many day in the month 31 days from now?

## May 
##  31

per <- period(hours = 10, minutes = 5) # define a period you can add to other date time...
per # a period class object

## [1] "10H 5M 0S"

Sys.time() + per

## [1] "2019-04-30 09:44:44 IDT"

Sys.time() - per

## [1] "2019-04-29 13:34:44 IDT"

difftime(Sys.time() + per, Sys.time(), unit = "hours")

## Time difference of 10.08333 hours

difftime(Sys.time() + period(week = 3), Sys.time(), unit = "day")

## Time difference of 21 days

exmaple_interval<- interval(start = dmy("01-04-2019"),end = dmy("01-06-2019"),tz="Asia/Jerusalem") # create time interval class
exmaple_interval

## [1] 2019-04-01 03:00:00 IDT--2019-06-01 03:00:00 IDT

today() %within% exmaple_interval

## [1] TRUE

quarter(Sys.time(), with_year = T) # the qurater of the year...

## [1] 2019.2

Part 2 - Taxi data frame

Import the data

taxi <- read_csv("taxi.csv")

## Warning: Missing column names filled in: 'X1' [1]

## Parsed with column specification:
## cols(
##   X1 = col_integer(),
##   taxi_id = col_integer(),
##   trip_start = col_datetime(format = ""),
##   trip_end = col_datetime(format = ""),
##   trip_miles = col_double(),
##   fare = col_double(),
##   tips = col_double(),
##   tolls = col_double(),
##   trip_total = col_double(),
##   payment_type = col_character(),
##   company = col_integer()
## )

Convert to time class (“POSIXct” or “POSIXt” )

class(taxi$trip_start)

## [1] "POSIXct" "POSIXt"

class(taxi$trip_end)

## [1] "POSIXct" "POSIXt"

taxi <- taxi %>% 
  mutate(trip_start = ymd_hms(trip_start, tz = "America/Chicago"),
         trip_end = ymd_hms(trip_end, tz = "America/Chicago"))

# If label is false then is will show the week day numbers (1:7)
wday(taxi$trip_start, label = TRUE) %>% head()

## [1] Wed Fri Sun Sat Thu Fri
## Levels: Sun < Mon < Tue < Wed < Thu < Fri < Sat

taxi %>% 
  mutate(week_day = wday(trip_start, label = TRUE)) %>% 
  ggplot(aes(week_day)) +
  geom_bar()

taxi <- taxi %>% 
  mutate(duration = difftime(trip_end, trip_start, 
                             units = "mins")) %>% 
  filter(duration < 100) #remove the abnormally long rides
  
taxi %>% 
  ggplot(aes(duration)) +
  geom_histogram(bins = 15) +
  xlim(-5, 100)

## Warning: Removed 2 rows containing missing values (geom_bar).

five_min <- as.period(5, unit = "mins") # create five minute object
taxi <- taxi %>%
  
  # if the duration is 0 add five minutes
  mutate(alt_end_time = ifelse(duration == 0,
                               trip_end + five_min,
                               trip_end)) %>% 
  # the output is in numeric format - change to datetime format and don't forget the time zone
  mutate(alt_end_time = as_datetime(alt_end_time, tz = "America/Chicago")) %>% 
  
  # fix the duration
  mutate(fix_duration = difftime(alt_end_time, trip_start, 
                                 units = "mins"))
taxi %>% 
  filter(fix_duration < 100) %>% 
  ggplot(aes(fix_duration)) +
  geom_histogram(bins = 15) +
  xlim(-5, 100)

## Warning: Removed 2 rows containing missing values (geom_bar).

Plots:

A basic plot

(p <- ggplot(taxi, aes(trip_start, fare)) +
  geom_point(color = "blue", alpha = 0.1) + 
  xlab("Trip start") + ylab("Trip cost ($)"))

To control the time axis use scale_x_datetime() or scale_x_date() if you only have date object

p + scale_x_datetime(date_breaks = "5 day", 
                     labels = date_format("%d-%m-%Y"))

The date_break argument controls for the tick mark

The labels argument controls for the label of your data

Note that the syntax here is different…

before each time variable, you need to add % sign, then the symbol for the time argument and the desired separator

link to the date symbol guide:

link[]

p + scale_x_datetime(date_breaks = "12 hours",
                     labels = date_format("%d-%m-%y %H:%M")) +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

How to show only some of the time series

start_time <- ymd_hms("2016-01-01 06:00:00", tz = "Asia/Jerusalem")
end_time <- ymd_hms("2016-01-05 06:00:00", tz = "Asia/Jerusalem")
# create a start and end time R object
start_end <- c(start_time, end_time)
p + scale_x_datetime(limits = start_end,
                     date_breaks = "12 hours", 
                     labels = date_format("%d-%m-%y %H:%M")) +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

## Warning: Removed 4406 rows containing missing values (geom_point).

By hour plots

If you want to plot hourly patterns there are 2 options

taxi <- taxi %>% 
  mutate(time = as.hms(trip_start)) 
ggplot(taxi, aes(time, fare)) +
  geom_point(color = "blue", alpha = 0.1) +
  scale_x_time(labels = function(x) strftime(x, "%H:%M"))

taxi <- taxi %>% 
  mutate(fake_date = ymd_hms(paste("2000-01-01", time)))
(p_2 <- ggplot(taxi, aes(fake_date, fare)) +
    geom_point(color = "blue", alpha = 0.1) + 
    xlab("Trip start (hour)") + 
    ylab("Trip cost ($)"))

p_2 + 
  scale_x_datetime(date_breaks = "2 hours", 
                   labels = date_format("%H:%M")) +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))