Working with Time Zones:
Everything You Wish You Didn't Need to Know

Paul Ganssle


Github repo for this talk
⚠️ Dialect Warning ⚠️

s/zee/zed/g

Introduction

UTC

  • Reference time zone
  • Monotonic-ish (what's a few leap seconds between friends?)

Time zones vs. Offsets

  • UTC+1 is an offset
  • Europe/London is a time zone
  • BST is a highly-context-dependent abbreviation:
    • British Summer Time (UTC+1)
    • Bangladesh Standard Time (UTC+6)
    • Bougainville Standard Time (UTC+11)

Complicated time zones

Non-integer offsets

Examples:

  • Australia/Adelaide (+09:30)
  • Asia/Kathmandu (+05:45)
  • Africa/Monrovia (+00:44:30) (Before 1979)

Change of DST status without offset change

  • Portugal, 1992

    • WET (+0 STD) -> WEST (+1 DST) 1992-03-29
    • WEST (+1 DST) -> CET (+1 STD) 1992-09-27

  • Portugal, 1996

    • CET (+1 STD) -> WEST (+1 DST) 1996-03-31
    • WEST (+1 DST) -> WET (+0 STD) 1996-10-27

Complicated time zones

More than one DST transition per year

  • Morroco, 2012
    • WET (+0 STD) -> WEST (+1 DST) 2012-04-29
    • WEST (+1 DST) -> WET (+0 STD) 2012-07-20
    • WET (+0 STD) -> WEST (+1 DST) 2012-08-20
    • WEST (+1 DST) -> WET (+0 STD) 2012-09-30

... and Morocco in 2013-present, and Egypt in 2010 and 2014, and Palestine in 2011.

Complicated time zones

Missing days

  • Christmas Island (Kiritimati), December 31, 1994 (UTC-10 -> UTC+14)
In [8]:
dt_before = datetime(1994, 12, 30, 23, 59, tzinfo=tz.gettz('Pacific/Kiritimati'))
dt_after = add_absolute(dt_before, timedelta(minutes=2))

print(dt_before)
print(dt_after)

1994-12-30 23:59:00-10:00
1995-01-01 00:01:00+14:00

Also Samoa on January 29, 2011.

Double days

  • Kwajalein Atoll, 1969
In [9]:
dt_before = datetime(1969, 9, 30, 11, 59, tzinfo=tz.gettz('Pacific/Kwajalein'))
dt_after = add_absolute(dt_before, timedelta(minutes=2))

print(dt_before)
print(dt_after)

1969-09-30 11:59:00+11:00
1969-09-30 12:01:00+11:00

Asia/Shanghai

Asia/Shanghai time zone map

Asia/Urumqi

Asia/Shanghai time zone map

Why do we need to work with time zones at all?

In [10]:
from dateutil import rrule as rr

# Close of business in New York on weekdays
closing_times = rr.rrule(freq=rr.DAILY, byweekday=(rr.MO, rr.TU, rr.WE, rr.TH, rr.FR),
                         byhour=17, dtstart=datetime(2017, 3, 9, 17), count=5)

for dt in closing_times:
    print(dt.replace(tzinfo=NYC))

2017-03-09 17:00:00-05:00
2017-03-10 17:00:00-05:00
2017-03-13 17:00:00-04:00
2017-03-14 17:00:00-04:00
2017-03-15 17:00:00-04:00
In [11]:
for dt in closing_times:
    print(dt.replace(tzinfo=NYC).astimezone(UTC))

2017-03-09 22:00:00+00:00
2017-03-10 22:00:00+00:00
2017-03-13 21:00:00+00:00
2017-03-14 21:00:00+00:00
2017-03-15 21:00:00+00:00

Python's Time Zone Model

tzinfo

  • Time zones are provided by subclassing tzinfo.

  • Information provided is a function of the datetime:

    • tzname: The (usually abbreviated) name of the time zone at the given datetime
    • utcoffset: The offset from UTC at the given datetime
    • dst: The size of the datetime's DST offset (usually 0 or 1 hour)

An example tzinfo implementation

In [12]:
class ET(tzinfo):
    def utcoffset(self, dt):
        if self.isdaylight(dt):
            return timedelta(hours=-4)
        else:
            return timedelta(hours=-5)
    
    def dst(self, dt):
        if self.isdaylight(dt):
            return timedelta(hours=1)
        else:
            return timedelta(hours=0)
    
    def tzname(self, dt):
        return "EDT" if self.isdaylight(dt) else "EST"

    def isdaylight(self, dt):
        dst_start = datetime(dt.year, 1, 1) + rd.relativedelta(month=3, weekday=rd.SU(+2), hour=2)
        dst_end = datetime(dt.year, 1, 1) + rd.relativedelta(month=11, weekday=rd.SU, hour=2)
        
        return dst_start <= dt.replace(tzinfo=None) < dst_end

print(datetime(2017, 11, 4, 12, 0, tzinfo=ET()))
print(datetime(2017, 11, 5, 12, 0, tzinfo=ET()))

2017-11-04 12:00:00-04:00
2017-11-05 12:00:00-05:00
In [13]:
dt_before_utc = datetime(2017, 11, 5, 0, 30, tzinfo=ET()).astimezone(tz.UTC)
dt_during = (dt_before_utc + timedelta(hours=1)).astimezone(ET())  # 1:30 EST
dt_after = (dt_before_utc + timedelta(hours=2)).astimezone(ET())   # 1:30 EDT

print(dt_during)   # Lookin good!
print(dt_after)    # OH NO!

2017-11-05 01:30:00-04:00
2017-11-05 02:30:00-05:00

Ambiguous times

Ambiguous times are times where the same "wall time" occurs twice, such as during a DST to STD transition.

In [14]:
dt1 = datetime(2004, 10, 31, 4, 30, tzinfo=UTC)
for i in range(4):
    dt = (dt1 + timedelta(hours=i)).astimezone(NYC)
    print('{} | {} |  {}'.format(dt, dt.tzname(), 
                                   'Ambiguous' if tz.datetime_ambiguous(dt) else 'Unambiguous'))

2004-10-31 00:30:00-04:00 | EDT |  Unambiguous
2004-10-31 01:30:00-04:00 | EDT |  Ambiguous
2004-10-31 01:30:00-05:00 | EST |  Ambiguous
2004-10-31 02:30:00-05:00 | EST |  Unambiguous

PEP-495: Local Time Disambiguation

  • First introduced in Python 3.6
  • Introduces the fold attribute of datetime
  • Changes to aware datetime comparison around ambiguous times

Whether you are on the fold side is a property of the datetime:

In [15]:
print_tzinfo(datetime(2004, 10, 31, 1, 30, tzinfo=NYC))          # fold=0
print_tzinfo(datetime(2004, 10, 31, 1, 30, fold=1, tzinfo=NYC))

2004-10-31 01:30:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h
2004-10-31 01:30:00-0500
    tzname:   EST;      UTC Offset:  -5.00h;        DST:      0.0h

Note: fold=1 represents the second instance of an ambiguous datetime

Comparing timezone-aware datetimes

In [16]:
dt1 = datetime(2004, 10, 30, 12, 0);   dt1a = datetime(2004, 10, 31, 1, 30)
dt2 = datetime(2004, 10, 30, 12, 0);   dt2a = datetime(2004, 10, 31, 1, 30)
dt3 = datetime(2004, 10, 30, 11, 0);   dt3a = datetime(2004, 10, 31, 2, 30)   # Unambiguous
  • Same Zone: Wall clock times are used, offset ignored
In [17]:
print_dt_eq(dt1.replace(tzinfo=NYC), dt2.replace(tzinfo=NYC))   # Unambiguous
print_dt_eq(dt1.replace(tzinfo=NYC), dt3.replace(tzinfo=NYC))
2004-10-30 12:00:00-04:00 == 2004-10-30 12:00:00-04:00: True
2004-10-30 12:00:00-04:00 == 2004-10-30 11:00:00-04:00: False
In [18]:
print_dt_eq(dt1a.replace(tzinfo=NYC), dt2a.replace(tzinfo=NYC))  # Ambiguous
print_dt_eq(dt1a.replace(tzinfo=NYC), dt2a.replace(fold=1, tzinfo=NYC), bold=True)
print_dt_eq(dt1a.replace(tzinfo=NYC), dt3a.replace(tzinfo=NYC))
2004-10-31 01:30:00-04:00 == 2004-10-31 01:30:00-04:00: True
2004-10-31 01:30:00-04:00 == 2004-10-31 01:30:00-05:00: True
2004-10-31 01:30:00-04:00 == 2004-10-31 02:30:00-05:00: False

Comparing timezone-aware datetimes

  • Different zones: If both datetimes are unambiguous, the absolute times are compared:
In [19]:
print_dt_eq(dt1.replace(tzinfo=NYC), dt2.replace(tzinfo=CHI))    # Unambiguous
print_dt_eq(dt1.replace(tzinfo=NYC), dt3.replace(tzinfo=CHI))
2004-10-30 12:00:00-04:00 == 2004-10-30 12:00:00-05:00: False
2004-10-30 12:00:00-04:00 == 2004-10-30 11:00:00-05:00: True

If either datetime is ambiguous, the result is always False:

In [20]:
print_dt_eq(dt1a.replace(fold=1, tzinfo=NYC), dt3a.replace(tzinfo=CHI), bold=True)
2004-10-31 01:30:00-05:00 == 2004-10-31 02:30:00-06:00: False

A curious case...

In [21]:
LON = gettz('Europe/London')

x = datetime(2007, 3, 25, 1, 0, tzinfo=LON)
ts = x.timestamp()
y = datetime.fromtimestamp(ts, LON)
z = datetime.fromtimestamp(ts, gettz.nocache('Europe/London'))
In [22]:
x == y
Out[22]:
False
In [23]:
x == z
Out[23]:
True
In [24]:
y == z
Out[24]:
True

Imaginary Times

Imaginary times are wall times that don't exist in a given time zone, such as during an STD to DST transition.

In [25]:
dt1 = datetime(2004, 4, 4, 6, 30, tzinfo=UTC)
for i in range(3):
    dt = (dt1 + timedelta(hours=i)).astimezone(NYC)
    print(f'{dt} | {dt.tzname()} ')

2004-04-04 01:30:00-05:00 | EST 
2004-04-04 03:30:00-04:00 | EDT 
2004-04-04 04:30:00-04:00 | EDT
In [26]:
print(datetime(2007, 3, 25, 1, 0, tzinfo=LON))

2007-03-25 01:00:00+01:00
In [27]:
print(datetime(2007, 3, 25, 0, 0, tzinfo=UTC).astimezone(LON))
print(datetime(2007, 3, 25, 1, 0, tzinfo=UTC).astimezone(LON))

2007-03-25 00:00:00+00:00
2007-03-25 02:00:00+01:00

Why it was non-transitive

In [29]:
print(f'x (LON):              {x}')
print(f'x (UTC):              {x.astimezone(UTC)}')
print(f'x (LON->UTC->LON):    {x.astimezone(UTC).astimezone(LON)}')

x (LON):              2007-03-25 01:00:00+01:00
x (UTC):              2007-03-25 00:00:00+00:00
x (LON->UTC->LON):    2007-03-25 00:00:00+00:00
In [30]:
print(f'y: {y}')
print(f'z: {z}')

y: 2007-03-25 00:00:00+00:00
z: 2007-03-25 00:00:00+00:00
In [32]:
x.tzinfo is y.tzinfo
Out[32]:
True
In [33]:
x.tzinfo is z.tzinfo
Out[33]:
False

Working with time zones

dateutil

In dateutil's suite of tzinfo objects, you can attach time zones in the constructor if you have a wall time:

In [34]:
dt = datetime(2017, 8, 11, 14, tzinfo=tz.gettz('US/Pacific'))
print_tzinfo(dt)

2017-08-11 14:00:00-0700
    tzname:   PDT;      UTC Offset:  -7.00h;        DST:      1.0h

If you have a naive wall time, or a wall time in another zone that you want to translate without shifting the offset, use datetime.replace:

In [35]:
print_tzinfo(dt.replace(tzinfo=tz.gettz('US/Eastern')))

2017-08-11 14:00:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h

If you have an absolute time, in UTC or otherwise, use datetime.astimezone():

In [36]:
print_tzinfo(dt.astimezone(tz.gettz('US/Eastern')))

2017-08-11 17:00:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h

pytz

In pytz, datetime.astimezone() still works exactly as expected:

In [37]:
print_tzinfo(dt.astimezone(pytz.timezone('US/Eastern')))

2017-08-11 17:00:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h

But the constructor or .replace methods fail horribly:

In [38]:
print_tzinfo(dt.replace(tzinfo=pytz.timezone('US/Eastern')))

2017-08-11 14:00:00-0456
    tzname:   LMT;      UTC Offset:  -4.93h;        DST:      0.0h

pytz's time zone model

  • tzinfos are all static offsets
  • tzinfo is attached by the time zone object itself:
In [39]:
LOS_p = pytz.timezone('America/Los_Angeles')
dt = LOS_p.localize(datetime(2017, 8, 11, 14, 0))
print_tzinfo(dt)

2017-08-11 14:00:00-0700
    tzname:   PDT;      UTC Offset:  -7.00h;        DST:      1.0h
  • You must normalize() datetimes after you've done some arithmetic on them:
In [40]:
dt_add = dt + timedelta(days=180)
print_tzinfo(dt_add)

2018-02-07 14:00:00-0700
    tzname:   PDT;      UTC Offset:  -7.00h;        DST:      1.0h
In [41]:
print_tzinfo(LOS_p.normalize(dt_add))

2018-02-07 13:00:00-0800
    tzname:   PST;      UTC Offset:  -8.00h;        DST:      0.0h

There is more detail on this on my blog post, "pytz: The Fastest Footgun in the West" (https://blog.ganssle.io/articles/2018/03/pytz-fastest-footgun.html)

Handling ambiguous times

Overview

Both dateutil and pytz will automatically give you the right absolute time if converting from an absolute time.

In [42]:
dt1 = datetime(2004, 10, 31, 6, 30, tzinfo=UTC)  # This is in the fold in EST

dt_dateutil = dt1.astimezone(tz.gettz('US/Eastern'))
dt_pytz = dt1.astimezone(pytz.timezone('US/Eastern'))
print(repr(dt_dateutil))
print_tzinfo(dt_dateutil)

datetime.datetime(2004, 10, 31, 1, 30, fold=1, tzinfo=tzfile('/usr/share/zoneinfo/US/Eastern'))
2004-10-31 01:30:00-0500
    tzname:   EST;      UTC Offset:  -5.00h;        DST:      0.0h
In [43]:
print(repr(dt_pytz))    # Note that pytz doesn't set fold
print_tzinfo(dt_pytz)

datetime.datetime(2004, 10, 31, 1, 30, tzinfo=<DstTzInfo 'US/Eastern' EST-1 day, 19:00:00 STD>)
2004-10-31 01:30:00-0500
    tzname:   EST;      UTC Offset:  -5.00h;        DST:      0.0h

dateutil

For backwards compatibility, dateutil provides a tz.enfold method to add a fold attribute if necessary:

In [44]:
dt = datetime(2004, 10, 31, 1, 30, tzinfo=tz.gettz('US/Eastern'))
tz.enfold(dt)
Out[44]:
datetime.datetime(2004, 10, 31, 1, 30, fold=1, tzinfo=tzfile('/usr/share/zoneinfo/US/Eastern'))
Python 2.7.12
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> from dateutil import tz
>>> dt = datetime(2004, 10, 31, 1, 30, tzinfo=tz.gettz('US/Eastern'))
>>> tz.enfold(dt)
_DatetimeWithFold(2004, 10, 31, 1, 30, tzinfo=tzfile('/usr/share/zoneinfo/US/Eastern'))
>>> tz.enfold(dt).tzname()
'EST'
>>> dt.tzname()
'EDT'

dateutil

To detect ambiguous times, dateutil provides tz.datetime_ambiguous

In [45]:
tz.datetime_ambiguous(datetime(2004, 10, 31, 1, 30, tzinfo=NYC))
Out[45]:
True
In [46]:
tz.datetime_ambiguous(datetime(2004, 10, 31, 1, 30), NYC)
Out[46]:
True
In [47]:
dt_0 = datetime(2004, 10, 31, 0, 30, tzinfo=NYC)
for i in range(3):
    dt_i = dt_0 + timedelta(hours=i)
    dt_i = tz.enfold(dt_i, tz.datetime_ambiguous(dt_i))
    print(f'{dt_i} (fold={dt_i.fold})')

2004-10-31 00:30:00-04:00 (fold=0)
2004-10-31 01:30:00-05:00 (fold=1)
2004-10-31 02:30:00-05:00 (fold=0)

Note: fold is ignored when the datetime is not ambiguous:

In [48]:
for i in range(3):
    dt_i = tz.enfold(dt_0 + timedelta(hours=i), fold=1)
    print(f'{dt_i} (fold={dt_i.fold})')

2004-10-31 00:30:00-04:00 (fold=1)
2004-10-31 01:30:00-05:00 (fold=1)
2004-10-31 02:30:00-05:00 (fold=1)

Handling imaginary times

dateutil

dateutil provides a tz.datetime_exists() function to tell you whether you've constructed an imaginary datetime:

In [53]:
dt_0 = datetime(2004, 4, 4, 1, 30, tzinfo=NYC)
for i in range(3):
    dt = dt_0 + timedelta(hours=i)
    print(f'{dt} ({{}})'.format('Exists' if tz.datetime_exists(dt) else 'Imaginary'))

2004-04-04 01:30:00-05:00 (Exists)
2004-04-04 02:30:00-04:00 (Imaginary)
2004-04-04 03:30:00-04:00 (Exists)

tz.resolve_imaginary

Generally for imaginary datetimes, you want to "skip forward" to what the the time would be if the transition had not happened. In dateutil you can use the tz.resolve_imaginary function to do this automatically:

In [54]:
dt = datetime(2004, 4, 4, 1, 30, tzinfo=NYC)
dt_imag = dt + timedelta(hours=1)
print(tz.resolve_imaginary(dt_imag))

2004-04-04 03:30:00-04:00
In [55]:
dt = datetime(1994, 12, 31, 9, tzinfo=tz.gettz('Pacific/Kiritimati'))
print(f'{dt} ({{}})'.format('Exists' if tz.datetime_exists(dt) else 'Imaginary'))

1994-12-31 09:00:00-10:00 (Imaginary)
In [56]:
tz.resolve_imaginary(dt)
Out[56]:
datetime.datetime(1995, 1, 1, 9, 0, tzinfo=tzfile('/usr/share/zoneinfo/Pacific/Kiritimati'))

Both of these functions work with pytz time zones as well.

dateutil's tzinfo implementations

UTC and Static time zones

In [64]:
# tz.UTC is equivalent to pytz.UTC or timezone.utc
dt = datetime(2014, 12, 19, 22, 30, tzinfo=tz.UTC)
print_tzinfo(dt)

2014-12-19 22:30:00+0000
    tzname:   UTC;      UTC Offset:   0.00h;        DST:      0.0h

Static offsets represent zones with a fixed offset from UTC, and takes a tzname or either number of seconds or a timedelta:

In [65]:
JST = tzoffset('JST', 32400)                       # Japan Standard Time is year round
IST = tzoffset('IST',                              # India Standard Time is year round
               timedelta(hours=5, minutes=30))  
EST = tzoffset(None, timedelta(hours=-5))          # Can use None as a name

dt = datetime(2016, 7, 17, 12, 15, tzinfo=tzutc())
print_tzinfo(dt.astimezone(JST))
print_tzinfo(dt.astimezone(IST))
print_tzinfo(dt.astimezone(EST))

2016-07-17 21:15:00+0900
    tzname:   JST;      UTC Offset:   9.00h;        DST:      0.0h
2016-07-17 17:45:00+0530
    tzname:   IST;      UTC Offset:   5.50h;        DST:      0.0h
2016-07-17 07:15:00-0500
    tzname:  None;      UTC Offset:  -5.00h;        DST:      0.0h

IANA (Olson) database

The dateutil.tz.tzfile class provides support for IANA zoneinfo binaries (shipped with *nix systems).

DO NOT USE tz.tzfile directly - use tz.gettz()

In [71]:
NYC = tz.gettz('America/New_York')
NYC
Out[71]:
tzfile('/usr/share/zoneinfo/America/New_York')

The IANA database contains historical time zone transitions:

In [72]:
print_tzinfo(datetime(2017, 8, 12, 14, tzinfo=NYC))      # Eastern Daylight Time

2017-08-12 14:00:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h
In [73]:
print_tzinfo(datetime(1944, 1, 6, 12, 15, tzinfo=NYC))    # Eastern War Time

1944-01-06 12:15:00-0400
    tzname:   EWT;      UTC Offset:  -4.00h;        DST:      1.0h
In [74]:
print_tzinfo(datetime(1901, 9, 6, 16, 7, tzinfo=NYC))     # Local solar mean

1901-09-06 16:07:00-0456
    tzname:   LMT;      UTC Offset:  -4.93h;        DST:      0.0h

tz.gettz()

The best way to get a time zone is to pass the relevant timezone string to the gettz() function, which is intended to be a Python equivalent to the TZ environment variable.

In [75]:
tz.gettz()      # Passing nothing gives you local time
Out[75]:
tzfile('/etc/localtime')
In [77]:
# If it finds a valid abbreviation for the local zone, returns tzlocal()
with TZEnvContext('LMT4'):
    print(gettz('LMT'))

tzlocal()
In [76]:
# Retrieve IANA zone:
print(gettz('Pacific/Kiritimati'))

tzfile('/usr/share/zoneinfo/Pacific/Kiritimati')

Timezone Tips

Civil time vs. Timestamp

  • Civil times are "wall times" - use this when it matters what the clock on the wall says (e.g. meetings, television shows)
  • Timestamps are specific moments in time: use this when it matters what order things happened in or the duration between two events (e.g. logging, astronomical events)

Store civil times as naive portion + time zone serialization (e.g. America/New_York). Store timestamps in UTC or equivalent.

IANA and CDR

  • Never rely on a 3-letter abbreviation.
  • IANA keys are lookups in the tz database
  • Use Unicode CLDR (Common Locale Data Repository) to get display names for time zones

Plug

Me

@pganssle / https://ganssle.io / https://blog.ganssle.io

dateutil

If you're interested in helping out with dateutil development, check out the issues on the github or e-mail me.

Open Issues - https://github.com/dateutil/dateutil - Check out the help wanted label!

Bloomberg

@TechAtBloomberg - https://www.TechAtBloomberg.com