Stock Predictions — Intraday Trading | by Monik Raj | Towards Data Science

Now let's have a brief idea about the problem set we are going to deal with.
Intraday-trading is that trading norm of the stock market whose vesting period is, in layman’s words, 1 day. Buyers buy shares at the opening time of market at a specific time window and then sell the same at the closing window of the same day. We are here dealing with a data set of one of the public companies of Tokyo, with daily data of two years from 2014–2016. We are going to use a simple machine learning model to :
Understand our dataMake predictions based on an algorithm
Understand our data
Make predictions based on an algorithm
For the reference, here is the link to my Github repository which contains :
Python script for performing analysisTest data in csv formatTrain data in csv format
Python script for performing analysis
Test data in csv format
Train data in csv format
github.com
Now since you can find all the python codes in my repository, I am not going to bore you by again taking you through the code base or explaining to you the same. The script is simple, stand-alone and pretty straightforward, so it’s quite simple to understand. I have also documented the script wherever required, and still if you are stuck, you can always drop a mail at bmonikraj@gmail.com and expect a reply within a day.
Once my professor at NIT Rourkela, India said, “Understanding of the data and the problem set is as important as, the statistical knowledge required to analyze the same”. We need to understand about intra-day trading and the aspects defining the behaviour of our target value.
So, our data set comprises of 7 columns, out of which 6 are features and 1 is target.
Features
Open — It is the opening price of a shares for that particular day
High — It is the highest price the shares have touched in throughout day
Low — It is the lowest price the shares have fallen to in throughout day
Close — It is the closing price of the shares for that day
Date — The date of the observation, mostly the index of our data
Volume — The number of shares sold in throughout day
Target
Stock Trading- The price of the total shares in the market for that day
Now, since our data has a ‘Time’ component, it is highly probable that our data is a Time-series data. But for a data to be qualified as time-series data, it must have a factor of either ‘Trend’ or ‘Seasonality’. To check if our data set is seasonal or trendy in nature, we can perform “Dickey-Fuller Analysis”. If our dataset is seasonal in nature, then we can analyze it as a ‘Time-series’ data.
While implementing Dickey-fuller analysis in Python, it requires a variable, ‘Frequency’. Frequency is mathematically defined as
Frequency = (Total time units expected till which data will show repetative patter, in minutes) / (time gap between each observations, in minutes)
Now, in our data-set, we have basically Intra-day trading data. That means, our data is based on everyday’s behaviours of market, incidents in market or industries, management changes in the company, interests of buyers, policies and such other governing factors. Thus, we can strongly believe that our data will not show any pattern for more than a day. And also, from data-set it is evident that our observation frequency is one day. So, we chose the frequency to be 1.
Here is the Dicker-Fuller Analysis report of my data-set
From the above graph, it is clear that our data is not seasonal in behaviour, which means our data set is not a ‘time-series’ data-set, since seasonality is 0.
Now, after removing the seasonality out of the equation, we have a data set which is completely numeric in nature and has five features, because ‘time’ is no longer a feature to be consider, since it is not time-series data.
We can find the graphs of behaviour of various features and target over time here
From the above, we can see the individual behaviour of the features with respect to time. We can also have the correlation graph between every feature with each other and understand the interdependent relation between every feature.
From the above graph, the diagonal matrix gives the histogram of the values each column, which is of no use for us since we are not interested in distibution of the features or target (as the number of variables are high and also they are correlated).
A general note on understanding correlation matrix plot
If the total graph seems like a curve (either linear or non-linear), then they are correlated to good extentIf the plot seems scattered, then they are not correlated as expected
If the total graph seems like a curve (either linear or non-linear), then they are correlated to good extent
If the plot seems scattered, then they are not correlated as expected
Now, we can infer from the above graph, that most of the features are inter-dependent of each other are correlated. ‘Open’ is highly correlated with ‘High’, ‘Low’ and ‘Close’. And ‘volume’ is correlated to a good extent with target, ‘Stock Trading’. Thus, all the features are important for analysis.
The problem can now be laid out in the simple terms as below :
Dataset has all numerical valuesNumber of Features = Dimension = 5One TargetTarget is ‘predicted’ (numerical)Train size < 1000
Dataset has all numerical values
Number of Features = Dimension = 5
One Target
Target is ‘predicted’ (numerical)
Train size < 1000
From the above list, it is the best option to choose linear regression model for the same because
It is a problem of prediction , so a regressor must be usedSince the dimension is quite small and so thus the train size, any complex and sophisticated model, like SVM regressor can overfit the model (You can try !)
It is a problem of prediction , so a regressor must be used
Since the dimension is quite small and so thus the train size, any complex and sophisticated model, like SVM regressor can overfit the model (You can try !)
After implementing linear regression from scikit-learn module of Python, we get the RMSE = 4253929991.85, after testing our model on test data
The value seems too high and seems a mis-fit, but we must not forget the fact that the order of features and target values are not similar or near to each other.
The order of values in ‘stock trading’ ( 1011) >> order of values in features (105 ~ 106)
So, from that manner, the RMSE’s order is justified, considering the order in which Mean Squared Error is calculated. Here is the plot of the Actual test target and calculated Test target. From this you can visualize how small the deviation is, hence the performance of our model.
Applying the knowledge of machine learning and algorithms to daily life scenario and better decision making is the main purpose of such academic advances. And after all, ‘numbers play an important role in decision making, because numbers never lie’.
If I may put a conclusion to my article at the end — “You don’t need a sword to cut your nails”
Reach out to me in case you have any queries, at bmonikraj@gmail.com
Note from Towards Data Science’s editors: While we allow independent authors to publish articles in accordance with our rules and guidelines, we do not endorse each author’s contribution. You should not rely on an author’s works without seeking professional advice. See our Reader Terms for details.