It’s official: time doesn’t exist | by Gonzalo Ferreiro Volpi | Towards Data Science

[IMPORTANT]: some people left the article thinking it was about phyisics. If you’re interested only in the ML part about how to treat time features, feel free to skip over the first part of the story and go straight to ‘Dealing with time features in our dataset’.
In December my girlfriend and I went back to Uruguay and Argentina, from where we are from. Over there, her grandfather, a man I personally find fascinating -in the best possible way-, recommended me a book peppering it telling me it was one of the hardest books he had ever read in his life. Let me explain myself: we’re talking about a super clever man who reads in between two and three books at the same time. He himself probably read more books than we can possibly imagine. But it didn’t stop there: before we left, he bought the book for me, so I left Buenos Aires around Christmas with a short book of around 150 pages in my hand called ‘The Order of Time’ by an Italian physicist called Carlo Rovelli.
He asked me to give him my opinion once I finish the book, so I read it consciously. I made sure I had understood every chapter and I highlighted all the book to read the most important parts again once I finish. I’m not ashamed of saying out loud that I had to read all chapters at least twice to get some understanding of them. The book starts by stating that if two twin brothers meet again after ten years, where one lived in the mountain and the other one in the plain, the former would be older since time passes faster in the mountain and slower in the plain. And this my friends, is no fairy tale: time indeed goes slower in places closer to the centre of the earth.
In short, the book explains that time as we know it and we have learnt to understand it doesn’t exist. And what really is time. Reading about all this, from a professional studying the subject since he was a college student, got me meditating about lots of stuff. I’m not going to bore you with self-reflection kind of thoughts, but the book caught me thinking also about who do we usually use time in machine learning and data science. Since in his book, Carlo states that time itself it’s nothing. What we perceive as the time it’s a human construction invented to measure how much time takes it to the earth to go around the sun. And that measure is done by metrics also invented by us like seconds, milliseconds, hours, minutes, etc.
Given this, for someone who doesn’t know what the concept of ‘minutes’ or ‘seconds’ is, who doesn’t know what day is today, and even worse, doesn’t know what ‘days’ or ‘months’, saying that now it’s 22:48 pm does not indicate anything at all. I know what you’re thinking: that kind of person just doesn’t exist. Well, even though that’s true, guess who doesn’t know all that: your machine learning algorithm. So the way we treat and feed time data to our model is key.
For all the curious minds, first I’m going to spend a few paragraphs to give a summary of the book. If you’re interested enough go ahead and read it. Otherwise, you can just skip over it. However, if you read it and you want to learn more, here a Ted Talk Carlo Rovelli gave 8 years ago, and the book it’s only around six pounds in Amazon.
As I said before, the book states that the concept of time as we know it doesn’t exist. Neither the metrics we use to measure it. And they don’t exist, because in the world of physics they don’t exist. The fundamental equations of the world do not have a variable for time. What does exist is a physical time, whose elapsing is not uniform but depends on many factors.
This physical time seems to not be a continuous-time, but a granular time, that goes jumping in-between values instead of elapsing. Where the notion of time under a specific value doesn’t exist in its more basic concept of time. Following this understanding of physical time, at a microscopic level time it’s made up of crashing atoms. Atoms that are scattered in a cloud of probabilities and we don’t have a precise way of knowing where are they going to appear. They appear and crash in between where the probabilities are good enough for it to happen. But when it does happen, that fraction of the universe goes from some specific somehow basic configuration to a more complex one. However, these particular configurations which allow that kind of microscopic events, depend explicitly on a blur or unfocused vision we have of the world. But this unfocused vision does not depend on some mental perception we might have, but instead, it’s a relative property. It is a property of an object in relation to another object. The concept of temporality then is deeply connected to this blur or unfocused vision of the world, where we ignore the most microscopic details of it. Therefore, this physical time is, ultimately, the expression of our ignorance about the world.
Regarding this last idea, it helped a lot to understand it a chapter which talks about the fact that a cat it’s not part of the elemental ingredients of the universe. It’s something more complex than that, that constantly appears in several parts of our planet. In a similar way, at the top of a mountain, we look at a valley covered with a blue sea of white clouds. If we climb the mountain, while we do it, the clouds stop being white; we find ourself in the middle of a fog. But the change is not instantaneous, it is gradual. The same happens with any kind of surface. A marble table would look like fog if could make ourselves small enough. In these examples, a real thing appears, things we are able to see and even sometimes touch. They appear from a world where at the most basic level there are no cats. In the same way, time arises from a world without time.
The world without time then, says the book, is nothing more than a network of interconnected events. Where the absence of time doesn’t mean that it’s frozen or static; it means that the happening it’s not sorted by a linear temporal line. If by time we understand only the happening, then everything is time. But at the same time, to my understanding, nothing is too. Or at least, nothing is past and nothing is future. We’re just a tiny happening of interconnected events in a universe that’s much bigger than we know and understand about it.
What we understand as the past and the future are within ourselves. We are the stories that live in our brain. Lines drawn from footprints of this constantly crashing of atoms. Lines that allow us to predict future events. Like for example, in which direction is going a ball that has been thrown to me and how to move our hand in order to catch it. This space, our memory, together with the constant practice of anticipation, is the source of our perception of time and of ourselves.
Now, as I said before, apart from all the self-reflection and deep kind of thoughts all this can spark, let’s take this down to the world of machine learning. We do have a memory to predict things like the example of the ball above. We do know that after January it comes February, that last year was 2019, or what minute comes after 23:59 pm. Also, industry-wise, our memory allows us to know, for example, that after the spring it comes the summer and that if I’m working for a company that sells beer, summer is probably a seasonal period for us, and therefore, we should see an increase in sales. At Ravelin, where I work, we might know that for some industries, certain hours or times of the day are more fraudulent than others. However, our machine learning model doesn’t know all that. And even if we feed it with the months of the year in a one-hot-encoding sort of way, the algorithm wouldn’t know that after December, it comes January. Or that after midnight the hour starts again from zero.
How should we treat timestamps then? Well, as I read in a Stackoverflow question a while ago:
You don’t really use timestamps as features because they wouldn’t be useful during the classification of unseen data. Imagine training a model with data obtained in 2018, and trying to classify data for 2019. The information is not on the dates but in the values of the other features!
Can you see the resemblance with what Carlo says in his book? As in real life, where time itself is nothing apart from references we might have in our memory to give context to it, the same happens with our machine learning model. Our time features only have value in relation with some other dates that might be relevant for whatever we are trying to predict.
According to this, values as the following wouldn’t make any sense for a machine learning model:
Datetimes like: 2019–12–24T10:25:11.577Z,
Or timestamps like: 1578830817’
How can we translate this into something useful for our algorithm? Well, at first we could make some simple transformation to give our model a better reference of how to compare in between rows or events happening at different datetimes or timestamps. For example, we could obtain:
month_of_the_year
day_of_the_year
day_of_the_week
hour_of_the_day
minute_of_the_day
Minute_of_hour
Pandas provides us with a neat table with lots of options from its documentation about Time series/date functionality:
By doing this, our model would know that August, the eigth month of the year, comes after July and before September. Or that in 2020 the 9th of May comes one hundred days after the 30th of January and one hundred days before the 17th of August.
We could even go further and do some other kinds of things. For example, we might know that Christmas is a sensible date for our business, and in fact, our model itself might be even able to find out that datetimes like ‘2019–12–24 08:57:04.927’ have something special/weird/peculiar going on while predicting. However, how can the model know that some other datetime comes exactly one day or twenty-four hours after that datetime? We need to make it easier for the algorithm to understand that. For this, we could set up features like:
is_christmas
days_since_christmas
weeks_since_christmas
Or if we are an online delivery company and we know that sales-wise our strongest moment of the week is Friday at 20:30 pm, we could create features like:
minutes_till_peak
minutes_since_peak
Again, we’re trying to give our machine learning model the same kind of context we have thanks to our memory and our understanding of time. However, I know what you might be thinking: ok, cool, but how am I supposed to deal with these features once I created them? Cause we would be transforming our timestamp into categorical features. And since we covered in a previous story of mine about categorical variables encoding, some algorithms don’t particularly enjoy dealing with this kind of features if we don’t feed them rightly. So what do we do?
Well, as mention before, we don’t want to do any sort of one-hot-encoding or similar transformation. If we do that we would be losing any value we might have in the continuity of the feature. In other words, the model would stop recognizing such things like that the sixth month of the year, June, comes after May and before July. However leaving them as they are presents another challenge: how can the model know that once December finishes, it comes January once again? Or that after the 23rd hour of the day it starts all over again from zero?
Lucky for us, there’s a very simple way of solving this: we could transform each variable into a two dimensions feature, using a sine and cosine transformation. Let’s go through an example.
Suppose you have a dataset with the feature ‘OriginalClientEventTime’ in it. At first, you would like to transform the type of the feature into Pandas Datetime, in order to afterwards obtain the hour of the day. You can do this as follows:
train[‘OriginalClientEventTime’] = pd.to_datetime(train[‘OriginalClientEventTime’])train[‘hour’] = [x.hour for x in train[‘OriginalClientEventTime’]]
In a similar way, you could do the same to obtain any of the properties listed above in the table from Panda’s documentation about Time series/date functionality.
Once we do this, these features would look the following way:
Now that we have the hour of the day, we can transform it into a two-dimensional feature in order for the model to pick up the nature of the original. We can easily do this in just two lines:
import numpy as nptrain[‘sin_hour’] = np.sin(2*np.pi*train.hour/24)train[‘cos_hour’] = np.cos(2*np.pi*train.hour/24)
And this is how our dataset would like now with the new features:
train[[‘OriginalClientEventTime’,’hour’,’sin_hour’,’cos_hour’]]
If we plot the sine or cosine features independently we wouldn’t notice much, but that’s because we need both variables to catch the proper nature of the feature. See now what happens when we plot both features at the same time on the Y and X-axis:
train.plot.scatter(‘sin_hour’,’cos_hour’)
Neat, right? The behaviour of time as we know it seems to be reflected in a perfect circle that loops in between the values of the hours in a day. We could do the same with any other time feature! The month of year, minutes of the day, day of the year, days till Christmas, etc.
As I told my girlfriend’s grandfather in my review of Rovelli’s book, surely there are tons of things from the text I didn’t understand. But I did get that if somehow we can remember stuff about the past, and predict things of the future -like where should I move my hand in order to catch a ball someone threw towards me-, is thanks to our memory and the concept of time we’ve built. A concept of time made up of metrics that only describe the distance in between events, like how long it takes for the earth to spin around the sun (which by the way, it’s 365.256 days).
So if we want our machine learning models to really learn, let’s try to teach them what we ourselves have learned over time.
Well, that’s all for today. Stay tuned for future stories featuring other kinds of techniques for dealing with categorical features. Meanwhile, the year is just starting, will you be looking for a job in 2020? Well, then you can’t miss my story about practical tips for landing your first data job. Or perhaps you’re still looking for the perfect Christmas gift to give yourself? Then check out this story about 9 data science-related books to read. If none of this is your case, feel free to visit my profile on Medium and check any other of my stories.
And if you want to receive my latest articles directly on your email, just subscribe to my newsletter :)
See you around! And thanks for reading!