3 tips to handle timestamps in C
By: Mario Emmanuel
Tags: C, Programming, Data, Time Series Analysis, Data Science

3 tips to handle timestamps in C

Image from Wikimedia. License CC BY-SA 4.0

Whoever has handled timestamps with timezones in time series analysis has always faced some headache. No matter how many times I deal with time series, which in my case invariantly involves dealing with time zones, I always have to revisit how it works because it is often counter-intuitive.

The need

Working with time series often involves dealing with different timezones. Even working with just one time zone, you might still want to visualise data in your local time zone, which might change depending on the country of the user.

This situation makes it convenient to use UTC as a reference. Why? Because it does not experience daylight saving changes and it is easy to convert to any timezone.

In terms of how can you represent a timestamp it can either be:

language specific date-time objects,

human-readable format based on integers (i.e. 20210126 for dates, 1530 for time),

human-readable text format (i.e. 1999–01–01T00:00:29.75),

UNIX Epoch time.

I prefer the latest because it simplifies the arithmetic of time. To iterate one minute you just have to increase 60 to whatever time cursor you are handling. It also defines a moment in time with precision. If you have hundreds of Gb or Tb of data, a purely numeric representation is the way to go.

I will present examples for this case, all the examples can be found in the documentation and Q&A forums — there is nothing new or exotic presented here — , but this information is often buried in other explanations, and often something as simple as converting a specific localised timestamp to UNIX Epoch and the opposite is not as simple to find as it shall be.

A timestamp requires a time zone

If you say, it is March 28th 2021 09:30 and you think you are done, you are wrong. The subsequent answer is, ok, it is March 28th 2021 09:30, but where? Every timestamp must have a location or time zone, implicit (can be by convention — all timestamps in this database use Frankfurt time — ) or explicit. So if you deliver data, do yourself and others a favour, do the exercise of explicitly stating the time zone of the timestamps included. This might be obvious but I assure you that more than half of the data sets I have seen involving timestamps lack this information.

A time zone is not the time at a given place

The first mistake I find when approaching the problem is that it is easy to mix concepts. You will often hear things like “Frankfurt is ahead 5 hours of New York”, or “New York time zone is EST” which are wrong.

Frankfurt is sometimes 5 hours ahead of New York but in others it is 6, and when that happens depends on the specific year you are dealing with.

New York is not EST, New York is EST half of the year, being the other half EDT. When the transition happens vary year to year and the transitions could even disappear in the near future.

EST is not Eastern Time but Eastern Standard Time (and definitely is not “Easter Time” as I heard once). If you want to say Eastern Time that is ET.

It is ok to make such mistakes as we all began there but those need to be fixed soon otherwise you will be doing the wrong calculations.

The simplest way I have found to deal with time series associated with a particular location (in my industry, financial data traded in a given exchange) is to deal with the actual region/city. So saying “time in New York” implies that it would be EST or EDT depending on the specific year and day of the year, but the timestamp is properly defined. It is also often the right way to store data as many events take place at a given time no matter what. Stock exchanges normally open at 9:30 every day and you do the school run for your kid at the same time in the afternoon every day, they do not modify it depending on if daylight saving is observed or not.

Specifying a time zone might not uniquely define a moment in time as during the daylight adjustments it might be not clear what is the exact moment in time you are specifying, but it happens that all daylight adjustments take place during a weekend. If your data lacks weekends (such as it happens in data coming from many business-related fields) then you do not care about this detail.

If the daylight change is important because your data can take place during the weekend, then you are forced to use UTC always, as it does not experience any changes. That is why the military uses Zulu time because it is a shared global way to specify a moment in time.

Tip 1: when you specify a timestamps always specify which specific location (city) that time belongs to. If your time series involve weekends, use UTC.

Using UNIX Epoch

The UNIX epoch time is defined as the number of seconds passed since January 1st, 1970 at 00:00 UTC. Notice that UNIX Epoch is UTC so it identifies without errors a specific moment in time. Never ask about the timezone of a UNIX epoch timestamp, it is UTC by definition.

A specific caveat about UNIX epoch are leap seconds, but unless you have to deal with leap seconds, UNIX epoch is fine. Bear in mind also that UNIX epoch can represent times up to 2038 timestamps when using 32 bits signed integers. See [1] for details, but normally these limitations will have no impact on your application.

Tip 2: UNIX epoch timestamps are a convenient way to store a given moment in time, they are normally accurate enough and they allow to do simple things such as adding 60 to get the next minute. They are efficient too as only 4 bytes can be used to store a timestamp. Being UTC by definition, there is no need to communicate the time zone and you always know how to convert that timestamp into any time zone.

The precision given by UNIX epoch is one second, although alternate version handling milliseconds are often used.

ANSI C

We will begin reviewing how C can handle timezones and timestamps. In C the standard never covered different timezones but it has somehow been covered in all UNIX systems since the 80s.

A UNIX Epoch time can be stored using a 32 bits integer, in the standard the type is time_t which has traditionally being an integer. The standard does not define a specific type (in my 64 bits FreeBSD box using clang is a long).

So if you want to specify a UNIX Epoch this is what you do:

/* 1616927900 Epoch is Sunday, 28 March 2021 10:38:20 GMT

1616927900 Epoch is Sunday March 28, 2021 06:38:20 (am) in time zone America/New York (EDT)

*/ time_t epoch_time = 1616927900;

In POSIX systems (all UNIX/Linux/BSD systems), you can convert this into any timestamp with timezone using TZ variable. Imagine that you want to know what time is that specific UNIX Epoch in New York. Then you do:

/* 1616927900 Epoch is Sunday, 28 March 2021 10:38:20 GMT

1616927900 Epoch is Sunday March 28, 2021 06:38:20 (am) in time zone America/New York (EDT)

*/ time_t epoch_time = 1616927900; /* CONVERT TO struct tm */

struct tm *converted_time;

converted_time = localtime(&epoch_time); /* GET SYSTEM TIME ZONE */

tzset(); /* DISPLAY TIME */

char buffer[26];

strftime(buffer, 26, "%Y-%m-%d %H:%M:%S", converted_time);

printf("Timestamp for %ld is %s according to system timezone.

", epoch_time, buffer); /* DISPLAY TIME ZONE */

printf("Timezone for %ld is %s.

", epoch_time, tzname[converted_time->tm_isdst]); --- $ TZ='America/New_York' ./test

Timestamp for 1616927900 is 2021-03-28 06:38:20 according to system timezone.

Timezone for 1616927900 is EDT.

The struct tm has a field named tm_isdst which can be -1, 0 or 1.

-1 is used when you specify a given timestamp (I will cover that case later), 0 will mean that daylight saving is not being observed and 1 means that daylight saving is being observed.

tzset will retrieve the specific zone of the computer, localtime will use it to fill tm_isdst when using localtime , so that field of the struct gives you the right struct. tzset also sets the extern array tzname which can be used to get the time zone name.

If you want to use a different timezone than the one you system uses you have two options:

you can precede your program with the right TZ variable: $TZ='America/New_York' ./my_program . This is not part of C but just how a UNIX shell works to customize the environment when invoking a command. you can explicitly set the TZ variable with putenv("TZ=America/New_York"); before invoking tzset .

If you want to do the opposite, i.e. getting the UNIX Epoch of a given localised timestamp (the question would be: which is the UNIX Epoch that corresponds to Sunday March 28, 2021 06:38:20 in New York?) this is the code you would use:

/* 1616927900 Epoch is Sunday, 28 March 2021 10:38:20 GMT

1616927900 Epoch is Sunday March 28, 2021 06:38:20 (am) in time zone America/New York (EDT)

*/ /* GET SYSTEM TIME ZONE, MUST BE NEW YORK */

tzset(); /* DEFINE THE TIMESTAMP struct tm */

struct tm localised_time;

localised_time.tm_year = 2021 - 1900;

localised_time.tm_mon = 3 - 1;

localised_time.tm_mday = 28;

localised_time.tm_hour = 6;

localised_time.tm_min = 38;

localised_time.tm_sec = 20;

localised_time.tm_isdst = -1;

time_t epoch_time = mktime(&localised_time); /* NOTICE HOW mktime FIXES tm_isdst */

printf("Fixed tm_isdst is %d.

", localised_time.tm_isdst); /* DISPLAY TIME ZONE */

printf("Timezone is %s.

", tzname[localised_time.tm_isdst]); /* DISPLAY EPOCH */

printf("Epoch timestamp is %ld.

", epoch_time); --- $ TZ='America/New_York' ./test2

Fixed tm_isdst is 1.

Timezone is EDT.

Epoch timestamp is 1616927900.

Notice that in the struct tm year is specified as the number of years since 1900 and that month is 0 for January. We use -1 for tm_isdst and mktime uses the time zone retrieved by tzset to fix this value and determine if it was EDT or EST at that day and time in New York.

This obviously only works if your computer is in America New York timezone or if you explicitly set or pass TZ to the program. Also notice that this will not work for Microsoft Windows.

Tip 3: In C UNIX/POSIX timezone resources and basic struct tm, mktime and localtime functions can be used to handle timezones. It works very well for most cases and it works with every C/C++ standard out of the box.

I use these simple recipes in C to handle timezones with success. There might be other cases where this simplicity might not be enough.

C++ has other alternatives to cope with timezones, the most relevant ones being the chrono [2] library part of the C++20 standard (the library can be also compiled for older standards). Boost [3] also includes a library to handle timezones, but apparently, it lacks some time zones or has limitations — I have not tried it — . There is also the ICU library [4] to handle localisation which seems to be exhaustive and complete and many other open source solutions. The problem with all those is that you add another dependency to your system and I did not find them straightforward to use.

I found the above method simple enough for my needs but it only works well for UNIX systems. It works for any C/C++ standard.

[1] https://en.wikipedia.org/wiki/Unix_time

[2] http://www.cplusplus.com/reference/chrono/

[3] https://www.boost.org/doc/libs/1_74_0/libs/locale/doc/html/dates_times_timezones.html

[4] http://site.icu-project.org/