fabiochiu/medium-articles huggingface dataset:
Data Visualization|| Seaborn:-
By: Sanjeev Kumar
Tags: Seaborn, Programming, Python, Data Science, Data Visualization

Photo by KOBU Agency on Unsplash

Hello Everyone, welcome again. In this blog, we are going to learn about Data visualization using Seaborn. So hold tight and let’s dive into it. One of the most important libraries for data visualization in python is “matplotlib” which we have discussed in my GitHub repository and the link is below hope you enjoy it because practical is better to understand that:- Click here

Data Visualization:-

Data visualization is the graphical representation of information and data with the help of charts, graphs, tables, and maps. With the help of data visualization, it is easy to understand the data better and understandable.

Seaborn:-

In Python, Seaborn is a library used for statistical graphics plotting for data visualization. It provides a high-level interface for drawing attractive and informative statistical graphics. It helps in a better understanding of the data. for installation, we need to run the below command in cmd.

pip install seaborn

Style and color:-

Styling and color are basically used for better visibility of the data so that it is better to understand.

We can also change the size of the graph and bars,

import seaborn as sns

import matplotlib.pyplot as plt tips = sns.load_dataset(‘tips’)

plt.figure(figsize =(12, 3)) #Use for changing

sns.countplot(x = ‘sex’, data = tips)

Distribution plot:-

Seaborn distplot lets you show a histogram with a line on it. This can be shown in all kinds of variations. We use seaborn in combination with matplotlib, the Python plotting module. The distplot() function combines the matplotlib hist function with the seaborn kdeplot() and rugplot() functions.

Categorical plot:-

A categorical variable is the one type that has more than two or categories. Like hair is a categorical variable having more number of categories like blonde, brown, red, etc. So to represent these we use several categorical plots mentioned below:

barplot

countplot

boxplot

violinplot

striplot

swarmplot

Matrix plot:-

It is used to plot matrix data, It is a color-coded diagram that has rows data, column data, and values. We can create a matrix plot in seaborn using the heatmap() function in seaborn.

Grid:-

It is basically used to plot several data in one figure so that it is easy to examine all the data in a single place.

We can also use the different plots for the different sections as given below.

Regression plot:-

The regression plots in seaborn are primarily intended to add a visual guide that helps to emphasize patterns in a dataset during exploratory data analyses. Regression plots as the name suggest create a regression line between 2 parameters and help to visualize their linear relationships. Regression plots in seaborn can be easily implemented with the help of the lmplot() function.
Exploring Numerai Machine Learning Tournament
By: Mohammad Olfatnia
Tags: Seaborn, Python, Data Science, Machine Learning, Numerai

Figure 10. Code for cross validation on a few classifiers

I chose LR, LDA, and XB algorithms based on previous figure and fitted them on the whole training dataset. AUC score of these three different model is between 0.516 to 0.517, and logistic regression is marginally better than the other two.

Figure 11. Logistic Regression, linear discriminant analysis, and gradient boosted tree classifiers on the training dataset.

Figure 12. roc_auc_scores for the 3 aforementioned classifiers.

Now that we know logistic regression is performing good on the dataset, let’s try to choose the best parameters for this classification model. There are mainly 2 parameters that we can change, penalty and C value. We have 2 option for penalty, L1(Lasso) or L2(Ridge) regularization. For C parameter, I chose 5 values [0.01,0.1,1,10,100] . A simple GridSearchCV suffices to find the best parameters that are ‘l2’ and C=0.01. With these new values we were able to increase AUC by 0.001 to 0.518!!!

Figure 13. Code to find the best parameter for logistic regression by GridSearchCV

We have 50 features in the dataset. Do we need all of them to achieve a high score? Let’s try to answer this question by performing dimensionality reduction on the training dataset. First, let’s start with principal component analysis (PCA). I created a pipeline and set n_components of PCA to [10, 30, 50]. I was surprised that result of GridSearchCV shows 10 components as the best answer; however, score has not change from what it was before and it is still 0.518.

Figure 14. Dimensionality reduction by principal component analysis (PCA) shows that 10 components are better than 30 or 50 components.

Prediction:

Ok! it’s time to predict on the validation dataset with the optimized parameters. Following image shows the code and the scores. AUC is 0.5237 and log loss is 0.6922. AUC=0.5237 is not very high, but as you can see in Figure 16, which shows the leaderboard page of Numerai website; reputation (the average of AUC over the last 20 rounds) is 0.5334 for the first person. Log loss of a classifier with predicted probability of 50% is -ln(0.5)=0.6931. 0.6922 log loss correlates to 50.05% predicted probability.

Figure 15. Code for fitting and predicting on the validation dataset

Figure 16. Leaderboard as of 6/18/2019. Reputation is the average AUC over the last 20 rounds.

Future Works:

Era column in the dataset represents a unit period of time, this could be a week, a day, or an hour, we don’t know. I ran cross validation on each of these eras, and compare the AUC score. It varies from 0.5 to 0.7. This is a large variance in AUC. I wish we had more information about the dataset and era column. Numerai suggests to “build a new model on the bad eras to combine with your main model. In this way, you may be hedged to the risk of bad eras” . I am not sure how to do that at the moment. But, I will try to figure it out. Any comments/suggestions are greatly appreciated!
Data Visualization in Machine Learning using Python Library
By: Shivansh Keshari
Tags: Seaborn, Matplotlib, Data Science, Data Visualization, Machine Learning

All the Beginner ,who are looking forward to work in the field of Data science , machine learning and Data Analytics . Data Visualization play a important to create a model for further prediction and analysis. As In the preprocessing we come know to about the data , but for better understanding of data and relation of different attribute between each other visualization is very useful.

What is Data Visualization?

Data visualization is a technique that uses an array of static and interactive visuals within a specific context to help people understand and make sense of large amounts of data. The data is often displayed in a story format that visualizes patterns, trends and correlations that may otherwise go unnoticed.

How used in Machine Learning?

Data visualization is an important skill in applied statistics and machine learning. Statistics does indeed focus on quantitative descriptions and estimations of data. Data visualization provides an important suite of tools for gaining a qualitative understanding.

Python as a programming language is widely used in field of Machine learning . It support with various libraries that we use in data visualization. and graph plotting

Matplotlib Seaborn Pygal Ggplot Plotly

From the above mentioned library , Now we will discuss and see the implementation of matplotlib and seaborn.

Installation of matplotlib and seaborn

Matplotlib : python -m pip install -U matplotlib (windows 10 operating system) Seaborn : pip install seaborn (windows 10 operating system) Dependencies:

1.numpy (>= 1.9.3)

2.scipy (>= 0.14.0)

3.matplotlib (>= 1.4.3)

4.pandas (>= 0.15.2)

Implementation of Matplotlib and Seaborn in Jupyter Notebook

As mentioned in the above image , firstly create a object for matplotlib library for plotting different type of graph. this object will call the different methods for visualization. Before further discussion let us see a magic function used with matplotlib.

%matlplotlib inline is called magic function. that we use in jupyter notebook for plotting the graph . This inline method call the object into the output screen from the backend.

We can create various type of plot with matplotlib such as bargraph , pie chart, scatter plot , Fill plot,etc.

Below images will show you how to implement few of those
Central Limit Theorem
By: Hoda Saiful
Tags: Central Limit Theorem, Seaborn, Data Analytics, Python

When analyzing data, studying a huge population as a whole may prove to be a tedious task. So instead, a sub-set of data points are drawn from the population, referred to as Sample.

Central Limit Theorem states that the mean of samples (size > 30) follows a normal distribution, no matter what the distribution of the underlying population is (exponential, uniform etc.)

Properties of Sampling Distribution

The mean of sampling distribution is close to the mean of the underlying population. As the size of the sample increases ; mean of the sample becomes “almost” equal to mean of the underlying population. This is because taking larger values removes sample bias by accommodating the diversity that is inherent in the population.

Depiction of Central Limit Theorem

Generate a random exponential distribution

Plot the mean of 1000 samples from the population. Size of each sample is 30

How does a sample size of less than or greater than 30 impact the distribution of Sample means.

As the sample size increases, the distribution of sample means resemble a normal distribution

As evident from the above diagram, the plot resembles a close to perfect normal distribution for sample size ≥ 30.
Visual Analysis of Indonesia’s Music Taste using API Spotify and Seaborn
By: Krisna Renaldi
Tags: Music, Seaborn, Python, Spotify

Analyze: The Artist

Data loaded:

sql = "SELECT name,followers,genre,one_genre,popularity,url FROM artists ORDER BY name ASC" df = pd.read_sql(sql,my_conn,parse_dates=True)

Fig 6: I focus on “followers” and “genre” field

I connected artist with number of followers.

Fig 7 there was no local artist

This is the local artist (in range million)/

Fig 8

Every artist brings their own genre. Luckily, Spotify has mapped genre on every artist (Fig 6). This chart genre vs follower.

Fig 9

It seems “dance pop”, “pop”, “k-pop” is the most wanted genre in Indonesia. Now, let us see genre in domestic/local music.

Fig 10

Finally, I found genre “dangdut” with 1.35 million followers compare to “indonesian pop” with 22.26 million followers. Now, it is starting to clear about Indonesia’s music taste.

Analyze: The Song

Spotify gave audio feature to all song as meta-data. This is the description:

Fig 11

The values of all the feature are lies in range 0–1 except “loudness” and “tempo”. I have extracted each song using Spotipy.

from sqlalchemy import create_engine sql = "SELECT track_id,uri,danceability,energy,loudness,mode,speechiness,acousticness,instrumentalness,liveness,valence,tempo FROM audio_features af ORDER BY id" df_track = pd.read_sql(sql,my_conn,parse_dates=True)

Fig 12

Then I plot all values to see the correlation.

import seaborn as sns sns.pairplot(df_track)

Fig 13

Please, forgive me about the image. You could save the image on your local computer and zoom in to see more detail. I noticed several things:

“energy” and “valence” are in linear relation.

“mode” could be dropped for this analyze.

“danceability” ,“energy”, “loudness” have positive correlation.

“acousticness” and “energy” have negative correlation and that is make sense.

To get more insight, I took top 100 song based on number of stream like I did in Fig 2 then I calculated average items in audio feature. Lastly, I calculated average items in audio feature for all songs. I plotted on radar chart.

Fig 14

There were almost no difference score between top 100 song and all song. Both had score 0.55 ~ 0.6. This was inline with Fig 9, Indonesia’s Spotify users tend to listen to song with positive ambience (“valence” score ~ 0.6) and dance intend (“danceability” score ~ 0.6). Also, with high energy.

Tips for composers and song producers, please make a song with “danceability”, “valence”, and “energy” score above 0.5 because this is what Indonesia wants!

Conclusion

After analyze the dataset, I concluded several things:
What is your AirBnB choice for next time you are in Sydney?
By: Madhuranga Rathnayake
Tags: Seaborn, Python, Udacity Nanodegree, Airbnb, Sydney

AirBnB publishes its data frequently at http://insideairbnb.com/sydney. As of August 2020, in the middle of a global pandemic, there were 31,000 AirBnB listings in Sydney. Let’s see what insights can we gain from those data.

What neighbourhood has most listings?

Sydney city itself is surrounded by a variety of attractions. Most of these attractions are located within walking distance from the CBD or a short light trail ride. Sydney city also has a higher number of high-rise apartments as compared to its suburban areas. For all these reasons, in 2020 Sydney city had the highest number of AirBnB listings followed by Waverly, Randwick, Warringah and Manly. The latter four neighbourhoods are very close to many popular beaches, resulting a higher rate of AirBnB listings.

Number of AirBnB listings by neighbourhood (a detailed graph can be found in Github)

Majority of the listings are either for a whole apartment or a private room. On average apartments can accommodate 4 guests while a private room can accommodate only 2 guests.

What is the average cost?

The next most popular question is how much these rooms cost? It is hard to comment on the accommodation prices since it purely depends on the needs and preferences of the individual/group. But, looking at the price visualisation below, a clear trend in prices can be seen. As you would naturally expect the price range increase from a shared room, which would be the lowest to a private room, then a hotel room — and an entire apartment being the most expensive. In general, Pittwater and Mosman have been highlighted as the most expensive neighbourhoods around Sydney city.

However, the average price per person per night in Sydney is significantly higher for a hotel room (73 AUD) than for a private room (31 AUD), shared room (28) and an entire apartment (24 AUD).
Build a Python project using Pandas and Seaborn | Eugene Tsai
By: Eugene Tsai
Tags: Seaborn, Visualization, Analytics, Python, Pandas

photo by Ian Parker

Introduction

Pandas is an open source Python library which is powerful and flexible for data analysis and manipulation. Seaborn is a Python library for data visualization which is built on Matplotlib. In this article, I select the penguins dataset from Seaborn built-in datasets for further analysis and visualization. By the way, I am a penguin lover and this is the primary reason why I choose the penguins dataset. If you are also a penguin lover or animal lover, do not forget to give me a clap and follow me!

Before start

We have to understand the body parts of a penguin in order to know the variables in penguins dataset.

photo by Maksim ŠiŠlo

Content

Import Pandas and Seaborn libraries

Select penguins dataset from Seaborn

Explore the penguins dataset

Clean penguins dataset

Visualization and analysis of penguins dataset

1.Import Pandas and Seaborn libraries

2.Select penguins dataset from Seaborn

There are eighteen datasets in the Seaborn library. Here, we select the “penguins” dataset.

3.Explore the penguins dataset

There are 344 rows and 7 columns in penguins dataset.

Looking through the first few rows in penguins dataset. Head function shows the first 5 rows in the dataframe by default.

Checking whether there are missing values in penguins. Here, we find out that there are some missing values in bill_length_mm, bill_depth_mm, flipper_length_,body_mass_g, and sex columns.

4.Clean penguins dataset

There are many strategies to handle missing values. Here, we replace the missing values with the most frequent value in sex column.

Seeing if there are still missing values. Look great!

5.Visualization and analysis of penguins dataset

Bar plot represents the total number of each type of penguins. Here, we find out that the largest population is Adelie.

Scatter plot reveals the relationship between the flipper length and body mass. Here, we find out that penguins with longer flippers are heavier.

Relational plot categorizes the relationship between the flipper length and body mass by sex. Here, we find out that male Gentoo penguins have the longest flippers and are the heaviest.

Box plot shows the minimum, maximum, median, first quartile, and third quartile of values in bill_length_mm, bill_depth_mm, and flipper_length_mm columns.

Violin plot depicts distributions of body mass of each type of penguins.

Pair plot shows the relationship between each variable. Here, we find out that we can sort out the species of penguins by their different characteristics.

Wrap up

There are still a lot of insights that can dig from the penguins dataset by using Pandas and Seaborn libraries. The penguins dataset is definitely the most entertaining and cutest dataset for analysis and visualization. Hope you can also get fun by analyzing the penguins dataset. Again, do not forget to give me a clap and follow me or connect me on Linkedin. Thank you for the reading!
Best News Website In Sri Lanka
By: Mohamed Zacky
Tags: Sri Lanka, News

In the highly competitive news broadcasting space, In Sri Lanka has built probably the most unbiased and comprehensive news property on offer “Sri Lanka C News” — with a news team of 100 people geared to deliver the most accurate and timely information. In Sri Lanka To be the number one News brand among Sri Lankans in terms of loyalty by reaching them through every possible opportunity in late 2020, In Sri Lanka entered into a partnership with Dialog Telekom, the largest mobile phone operator in Sri Lanka, to send breaking news alerts to all Dialog mobile subscribers as Short Text Messages (SMS). Currently, instant news alerts can be obtained through all the leading operators.

Today, In Sri Lanka is characterized by its availability across delivery channels such as Television, Radio, Web, SMS alerts, IVR, Video Alerts, and social media such as Facebook, Twitter and YouTube.

In most of these forms, In Sri Lanka is available on a multilingual platform providing greater accessibility to consumers both in Sri Lanka and across the world. The style of the news property has been customized to suit each of the delivery media, taking note of the nature of consumption.

The showpiece of the In Sri Lanka news property is the in-srilanka.com web portal which is ranked the Number One news site used by Sri Lankans; analysis shows that surfers spend more time on the site than they do on that of the nearest competitor. Available in Sinhala, Tamil and English, the site facilitates two way viewer generation by offering consumers an interactive platform to comment on the latest stories and interact with the In Sri Lanka community, and by allowing viewers to file in their own news reports through Newsdesk — My Report.
Cosmopolitanism theory
By: Yovindu Walpola
Tags: Sri Lanka, Easter Sunday Attacks, Politics, Cosmopolitanism

Cosmopolitanism theory

Cosmopolitanism theory which emphasizes that all people are entitled to equal respect and consideration no matter what their citizenship status or other affiliations happen to be is more of a logical political approach to living in harmony with different ethnicities and groups of people. It has many practical loopholes but the overall bottom line idea put forth a strong advocacy in favor of equality.

Easter bombing is a primary example in showing how this cosmopolitanism theory was exploited in order to create chaos among religious groups for mutual benefits by an individual party. Cosmopolitanism theory is not a law that could be ordered to follow but more of an ethic which is already within a certain portion of the society. The very idea of cosmopolitanism which suggests each and every person is entitled to equal respect was the same root cause for the dispute among religions. Equality and respect is relative. One should be worthy of respect. If we really do believe in a cosmopolitanism theory in Sri Lanka, our main language wouldn’t be Sinhala. There won’t be any minorities. We won’t be calling ourselves a Sinhala Buddhist country. It is not practical to follow it. It is more of a theory put forth by the majority to make the minority to feel like they are still mattered. There is no 100% equality or respect in anywhere of the world. When divided by groups in according to one’s ethnicity or religion, it is not possible to maintain a theory like cosmopolitanism as each individuals within the groups may have different mentalities. When acting as a group each individual action has an impact on the overall group. This is the main reason which has led to create social stereotypes among people.

Acts of separatism, extremism and racism will always be present within the society but it is possible to manage the general public if they are educated enough to differentiate between the brainwashing and faith. An educated citizen who questions authority is a hundred times better than an average individual who believe in a theory for the sake of believing.
Meanwhile in SL as Empowered Youth
By: Meanwhile In Sl
Tags: Meanwhile In Sl, Sri Lanka, YouTube, Youth

Meanwhile in SL is one of the most popular youth-run YouTube channels in Sri Lanka. By today, after two years since the day it was founded, they have passed 2, 75,000 subscribers, a number accounting for both national and international fans. The Meanwhile in SL crew consists of 10 members, namely, Dedunu Akarshanie, Fairoza Fairooz, Sulakkhana Herath, Kanchuka Samarakoon, Lahiru Keerthirathne, Manula Hewage, Shashika Milantha, Yasas Kulasekara, Kavinda Sumanaweera and Isuru Ranaweera.

One of the most common criticisms aimed at the youth nowadays is that they are not consistent. It is a common belief that the new generations lack the motivation to continue a task with the same interest and commitment for a long time. The Meanwhile in SL team proves otherwise. They have been in the business of creating YouTube content for over two years, and despite their personal careers and higher education affairs, they are becoming increasingly popular on the world’s second most visited cyber platform –YouTube.

Through their production, these young and ambitious YouTubers are working towards breaking down the stereotypes that exist in society in a subtle way. The concepts of their videos are by and large based on real life experiences and characters. The Meanwhile in SL crew through their creative scripts makes an attempt not only by illustrating our daily lives as it is but also by making the viewers question certain norms and practices that have been instilled in society.

As a collective effort of an energetic and goal-oriented team, Meanwhile in SL produces high quality and family friendly output on a regular basis. Diverse as they are in their own ways, they work in perfect unity to spread laughter not only within Sri Lanka, but also to their international audience. These empowered youth are multi talented and humble, which is evident by the growing fan base they already have. People embrace Meanwhile in SL not merely as a YouTube channel, but also as a group of friends who have devised a way to spend time together balancing both their personal and professional lives. They are adored and celebrated for the unique bond they share.

Meanwhile in SL encourages young individuals to think creatively and to work in team spirit. They are continuously learning, earning and they are most importantly enjoying what they do. Knowingly or unknowingly they definitely inspire youth to take a responsible initiative about their lives. In other words, they are positively empowered youth, and they continue to share that empowerment with others which is the kind of passion we need right now.
What You Seek is Seeking You! — Rumi
By: Girish Borkar
Tags: Spirituality, Spiritual, Mindfulness, Spiritual Growth, Meditation

The universe is always waiting to help us reach our goals. The trouble with us is that we keep changing our goals. When we go to management school to set short, medium and long term goals; or even if we go for a job interview, we are asked — what are your goals — please briefly state your short, medium and long term goals. This allows the interviewer to assess whether we will be able to fit into the organisation culture and whether our goals are more or less in alignment with the organisation goals. This is fine when it comes to jobs — but what about our life goals? Have you ever defined them? Have you ever in your wildest dreams had liberation (moksha) as one of your goals? Because, if that is not your ultimate goal then you have missed the purpose for which you have taken this birth!

From the moment we take birth the universe is conspiring to take us to our ultimate life goal that is liberation. We lose our path — in this competitive world our parents feel that earlier the child starts learning the better (we have schools for 1.5 to 2 years old toddlers) — education starts that early — meaning we have already started going contrary to the universe’s goals! Even then the universe keeps sending messages to each one of us, but we keep missing the messages or misinterpret them to suit our own ideas of life and we get stuck in this birth-death-birth cycle!

Over several births and several Gurus, we ultimately reach a birth which could be said to be our last birth — this is after we have lived all our dreams, satisfied most of our desires and realised for yourself that we are not happy in spite of having everything that life can offer. That is when we start looking for spiritual guidance, start turning inwards. Once we do that, we actually start correctly interpreting the messages that the universe is sending us. The Guru-energies then direct us to a realised Master who becomes the inspiration for us to turn inwards. Once you turn inwards, you find peace and solace which all the material wealth, education, power, post-position could not give you all these years. You then slowly start understanding the world of illusion in which we live, one realises that what we had been seeking all our lives had been seeking us over several births. We ultimately find that divinity, peace, silence, joy and bliss right within us while we had been chasing all of this outside in the material world. Ultimately, we find our own true self and become one with nature! Self-realisation at last!
Understanding Spiritual Vibrations
By: Girish Borkar
Tags: Spirituality, Spiritual, Mindfulness, Spiritual Growth, Meditation

Vibration or Universal Life Energy is simply a method to describe your overall state of wellbeing. Everything in the universe is made up of energy vibrating at different frequencies. The vibration is caused by “porous breathing” of the entire body. You are simply opening up your channels and allowing the intake of energy from around you. The spinal feeling is what we call the opening of the back channel. It’s simply the movement of bodily energy.

The first time I felt vibrations was during my first meditation camp in February 2009 — on the day of awakening Swamiji says — some of you will feel cold vibrations on the palms of your hands — the moment he said that there were two eddies of ice-cold energy whirling on both the palms and I was wondering what kind of magic is this. Then Swamiji goes on to say those who felt cold vibrations are in a good spiritual state. After that, with regular meditation — both morning and evening as I got pulled into spirituality and started enjoying these vibrations — I also started getting floral fragrances — sometimes rose, jasmine, sometimes sandalwood, and that of havan.

All these fragrances start coming from within you as your chakras start getting cleared. When you get these fragrances during meditation, enjoy them, don’t doubt them and start looking everywhere to see if someone has put on perfume or something! The moment you doubt the fragrance, it will be a long time before you get it again. Faith and acceptance are two key words we need to adopt in our life, if we want to progress spiritually.

Energy and vibrations are something that you start receiving as you make spiritual progress one step at a time. Some people feel the vibrations pretty early — that is because they probably have been born with a bank of spiritual savings from the previous birth and this birth is a continuance of that journey. The vibrations could start from the crown chakra — circulating around the crown for some time, before starting to move downwards — I have always gone into meditation the moment the vibrations went below the throat chakra — the cool vibrations are so soothing as to put you almost immediately into a drowsy meditative state, after which you are lost to the world.
Remaining Balanced in an Imbalanced World
By: Girish Borkar
Tags: Spirituality, Spiritual, Mindfulness, Spiritual Growth, Meditation

These days the world is seeing a lot of upheaval — with the corona virus spreading all over the world and mutating and changing form — giving rise to questions on the efficacy of the available vaccines. Then we have cyclones causing their devastation, human beings adding their bit by going to war causing massive destruction. So, is this nature’s way of getting back at us? Or is the churn really on now (I had mentioned that the churn has begun in a March 2020 post) — because globally the intensity has gone up manifold. The poison in the system is slowly coming out — and as per tradition the poison has to completely come out before the nectar can be experienced.

We are living in times of total stress, people very close to us are being affected, people are dying and there is a feeling of helplessness which is pervading in society, there is anger at the system failing its citizens. But if you sit back and witness whatever is happening you will realise that nature is trying achieve a balance and balance can only be achieved when there is large scale devastation — that is nature’s law — creation always takes place after destruction. It appears that we are in such a phase now. The question is how to remain balanced in this imbalanced world?

I can only tell you what has worked for me, it is not necessary that it will work for you. But you can definitely try out variations on similar lines. I have stopped watching the news on TV for the last ten years now — because the news is always negative, because negativity sells in this morbid world — that is what gets you the ad revenue! I read the newspaper headlines and hold far more interest in solving sudoku, hitori, 7 little word puzzles as it keeps the brain cells active.

Meditation has helped me live in the present moment, helped me in trying to be a witness — I am getting there, the past has become history, the future is a mystery — so living in the present I found is very comforting, joyous and the only reality. Writing positive notes has kept me preoccupied, office work gets done in the ‘now’ — leaving nothing for tomorrow. The days just go by in harmony, filled with positive energy and love. I find that this stay at home has become more of a stress reliever and helped me finding out more dimensions about myself. I am sure you guys too can try this or find something similar that works for you. Live in the moment and you will be fine!
Is Life just about Emotions?
By: Girish Borkar
Tags: Spirituality, Spiritual, Mindfulness, Spiritual Growth, Meditation

We react emotionally to situations throughout our lives — but that is not just true of human beings but also of the animal world. There are so many stories, where dogs, cats, elephants, crocodiles, etc. show their sensitive sides and react emotionally by denying themselves food when someone close to them dies or falls sick.

Rational thought helped us shape the world for our future: rerouting rivers, breeding plants, caging tigers. We now have two highly developed systems: reason and the emotional core still sitting there, like the primitive animal inside us. As said above, mental illnesses often result from an imbalance in those two.

The distinction between man and animal is that man has got the capacity to think whereas animals do not have that capacity. From very early in life, in fact immediately after a child is born love flowers between mother and child, then if the child falls sick there is care and concern, which turns to worry and fear if things turn for the worse. Therefore, it has always been said that man is a bundle of emotions. Being emotional is not bad, but invariably extreme emotional reactions lead to all round stress.

If we learn to manage our emotions, get control over our emotions then we can be both loving and compassionate which are the positive manifestations of our emotions. It is necessary to master the negative emotions, better still it would be wonderful if we could just focus on the positive aspects of life and grow emotionally strong.

The way to mastery over emotions is through the practice of mindfulness and meditation. Meditation under the guidance of a realised Master helps us turn inwards and our search for external happiness comes to an end. The inward journey takes us to the core where our identity merges with that of the soul. Once that happens, we become a witness to life — becoming a witness means transcending the emotional state of human beings and becoming loving and compassionate towards all those whom we come in touch with. So, even though life is about emotions — what is important is how we enhance our positive emotions while mastering the negative ones!

Link to a healthy weight loss diet — http://shop.zoe.menu/VB
Courage and Faith
By: Girish Borkar
Tags: Spirituality, Spiritual, Mindfulness, Spiritual Growth, Meditation

The dictionary definition of courage is the ability to do something that frightens one, the strength to face pain or grief. We act brave and courageous throughout or life both knowingly and unknowingly. When we face down our fear of people or things, when we say sorry — yes, it requires tremendous courage to own up one’s mistake and say one is truly sorry — the ability to speak up and say what is right, specially to superiors and not be a yes man; the courage to keep quiet and remain silent when the need arises; the courage to recognise and acknowledge the character of other individuals; courage to stand one’s ground and not give in when you know you are right. There are lots of dimensions to courage.

Faith is defined as trust or confidence in someone or something; strong belief in the doctrines of a religion, based on spiritual conviction rather than proof. We need faith and belief to get through a lot of situations that suddenly crop up in life. Whether it is a family crisis, a job crisis, loss of self-belief, a financial crisis and so on. When one faces these situations in life we usually turn to God — it is human nature to turn to the divine when we are in trouble. Invariably when things are good, we do not remember Him, only troubles lead us to His door.

When we start meditating and turning inwards — our inner journey takes us down an unknown, untraversed path. Here too one requires courage and faith — the courage to face the fear of death and go beyond and the faith that you will surmount this fear and reach your goal. As you go within, you start losing your ego, your material desires start disappearing, you become more accepting and with this your courage gets a boost up as you become bluntly honest, no beating around the bush and simultaneously you develop the faith to face all situations in life with a smile. You develop a confidence in your SELF — you become an epitome of calmness and peace which radiates from your very being. Courage and faith go hand in hand — meditate regularly and develop these qualities to such an extent that your aura radiates your confidence and love towards all beings — both sentient and non sentient!

Link to a healthy weight toss diet — http://shop.zoe.menu/VB
Why to help Seattle Airbnb hosts and guests make smarter decisions?!
By: Rahil Bagheri
Tags: Time Series Forecasting, Airbnb Seattle Open Data, Data Science, Machine Learning, Prophet

What are the key drivers of higher rates?

The average listing price for 2016 was about $137 per night. But it varied through different months and weekdays. It even reaches close to 160 during summer. Seattle is a cold place, and it makes sense to be more expensive during summer due to higher demands.

It also seems like the listed places on Seattle Airbnb during weekends are more expensive than other weekdays.

So, it is statistically significant that time and market are the main drivers and should be considered as a regressor in Machine Learning models. Houses and apartments were the most listed and wanted properties but with reasonable prices, less than $150 per night. But, some features like boat, loft, condo, and townhouses were expensive and rarely listed.

To identify the drivers of higher ratings, I cleaned, scaled, and formatted the data ready for predictive analysis, also removed the variables that have similar performances, multicollinearity. Then used Random Forest regressor to find most significant listing features to explain what caused the price range variation.

Random Forest selects the critical features by measuring two performance metrics, decrease node impurity, and increase accuracy. All the possible price drivers are given scores. The higher the weight is, the more significant the effect of listing feature is differentiating price values. The details about how Random Forest permute the values of each element is beyond the scope of this project.

First, I fitted the Random Forest model and then enhanced it by tuning the model hyperparameters to reach the highest accuracy.

The result of this proposed solution shows that the following features were contributed to making the critical decision in the tress and can explain 90% of the variation in price range:

Essential elements in this regressor were number of bedrooms, number of bathrooms, number of beds, number of included guests, number of reviews, season, neighborhood, cancellation fee, and policy strict.

In general, these features can be categorized into three broad categories; one is the properties features, previous reviews, and seasonal and weekly demand which I analyze more in the next part.
Strategic Analysis of Trump Rallies with NLP and Time Series
By: Tan Pengshi Alvin
Tags: Time Series Forecasting, NLP, Data Science, Trump, Donald Trump

Strategic Analysis of Trump Rallies with NLP and Time Series

Photo by Matt Smith on Shutterstock

The United States Presidential election will be decided on 3 November 2020, with highly reverberating effects on America and the world. While some people see Trump as a good president, others see him as a populist. Regardless of one’s political inclination, for good or worse, the consequences of the 2020 election could potentially be earth-shattering.

Nonetheless in this article, I will present to you a strategic analysis of Trump rallies, playing as a de facto data scientist on the side of the Biden-Harris campaign. I will attempt to topic-model the rallies with Natural Language Processing (NLP), and then I will identify topic trends in the rallies with time series, before offering a strategic analysis. Firstly, I obtained transcripts of Trump rallies from https://factba.se/transcripts.

1. Text Pre-processing

In total, I obtained data of about 100 Trump rallies transcripts from his first rally (Melbourne, Florida) in 2017, up till the first rally (Sanford, Florida) after Trump recovered from Covid-19 on 12 October 2020. Thereafter, I cast the data into a Pandas data frame with the Venue, Date and Transcripts as columns. Below is an excerpt from Trump’s rally in Sanford, Florida, and immediately we see the need for text processing:

‘Hello, everybody. Hello, Orlando. Hello, Sanford. It’s great to be with you. Thank you. It’s great to be back. [Audience chants “USA”] That’s a lot of people. You know our competitor, sleepy Joe, he had a rally today and practically nobody showed up. I don’t know what’s going on. Sleepy Joe, but it’s great to be back in my home state, Florida to make my official return to the campaign trail. I am so energized by your prayers and humbled by your support. We’ve had such incredible support and here we are. It’s you know, here we are. But we’re going to finish, we’re going to make this country greater than ever before. [Audience chants “We love you”] Thank you. Thank you. Thank you very much.

With regular expressions, I removed the audience inputs (in square brackets) from the transcripts. I further used NLTK’s RegexpTokenizer to tokenize words with 3 or more letters and removing words with any punctuation, before joining them again into a processed/fragmented transcript.

2. EDA — Trump’s Top 50 Words

With a ‘cleaned’ set of transcripts, I proceed to begin modeling. On exploratory data analysis, I used WordCloud to generate the top 50 words that Trump often used in his first 2 rallies and the last 2 rallies that I obtained:
Exponential Smoothing with Python
By: Nicolas Vandeput
Tags: Time Series Forecasting, Time Series Analysis, Demand Forecasting, Python, Forecasting

Do It Yourself — Simple Exponential Smoothing with Python

Simple smoothing function

We will define a function simple_exp_smooth that takes a time series d as input and returns a pandas DataFrame df with the historical demand, the forecast, and the error. The function also takes extra_periods as an input, which is the number of periods that need to be forecast into the future. The last input is the alpha parameter.

def simple_exp_smooth(d,extra_periods=1,alpha=0.4):

d = np.array(d) # Transform the input into a numpy array cols = len(d) # Historical period length d = np.append(d,[np.nan]*extra_periods) # Append np.nan into the demand array to cover future periods f = np.full(cols+extra_periods,np.nan) # Forecast array f[1] = d[0] # initialization of first forecast # Create all the t+1 forecasts until end of historical period for t in range(2,cols+1): f[t] = alpha*d[t-1]+(1-alpha)*f[t-1]

f[cols+1:] = f[t] # Forecast for all extra periods

df = pd.DataFrame.from_dict({"Demand":d,"Forecast":f,"Error":d-f})

Playing with our Function

We can then simply call our function (here with a dummy demand time series):

import numpy as np

import pandas as pd d=[28,19,18,13,19,16,19,18,13,16,16,11,18,15,13,15,13,11,13,10,12] df = simple_exp_smooth(d,extra_periods=4)

You can also quickly get the forecast accuracy with simple commands, such as shown below. See my previous article about KPI to see which one is best for your case.

MAE = df["Error"].abs().mean() print("MAE:",round(MAE,2)) RMSE = np.sqrt((df["Error"]**2).mean()) print("RMSE:",round(RMSE,2))

You should get these results:

MAE: 2.74 RMSE: 3.89

Another exciting step is to plot the results to analyze how the model behaves.

df.index.name = "Periods" df[["Demand","Forecast"]].plot(figsize=(8,3),title="Simple Smoothing",ylim=(0,30),style=["-","--"])

You should get a figure similar to the one below.

About the Author

Nicolas Vandeput is a supply chain data scientist specialized in demand forecasting and inventory optimization. He founded his consultancy company SupChains in 2016 and co-founded SKU Science — a fast, simple, and affordable demand forecasting platform — in 2018. Passionate about education, Nicolas is both an avid learner and enjoys teaching at universities: he has taught forecasting and inventory optimization to master students since 2014 in Brussels, Belgium. Since 2020, he is also teaching both subjects at CentraleSupelec, Paris, France. He published Data Science for Supply Chain Forecasting in 2018 (2nd edition in 2021) and Inventory Optimization: Models and Simulations in 2020.
Simple exponential smoothing with Excel
By: Nicolas Vandeput
Tags: Exponential Smoothing, Time Series Forecasting, Time Series Analysis, Supply Chain, Forecasting

Do It Yourself — Simple Exponential Smoothing with Excel

In this article, you will implement, in excel, a simple exponential smoothing forecast model.

1. We start our data table by creating three columns:

Date on column A

Demand on column B

Forecast on column C

2. Next to this, let’s add a cell with alpha (F1 in our example). Don’t forget to clearly indicate that this cell is alpha.

3. Once this is done, you can initialize the first forecast (cell C2) as the first demand (cell B2). We used dummy values here.

4. We can now populate our forecast; as of cell C3, we can use this formula:

5. To continue our forecast until the end of the historical period, we can simply drag this formula until the end of the table.

6. All the future forecasts (i.e., the forecasts out of the historical period), will simply be equivalent to the very last forecast based on historical demand (as shown below).

That’s already it. You can find more advanced models on my blog here, or in my book here.

You can also implement the simple exponential smoothing in python (here).

About the Author

Nicolas Vandeput is a supply chain data scientist specialized in demand forecasting and inventory optimization. He founded his consultancy company SupChains in 2016 and co-founded SKU Science — a fast, simple, and affordable demand forecasting platform — in 2018. Passionate about education, Nicolas is both an avid learner and enjoys teaching at universities: he has taught forecasting and inventory optimization to master students since 2014 in Brussels, Belgium. Since 2020, he is also teaching both subjects at CentraleSupelec, Paris, France. He published Data Science for Supply Chain Forecasting in 2018 (2nd edition in 2021) and Inventory Optimization: Models and Simulations in 2020.
Time Series Forecasting — A Getting Started Guide
By: Oscar Arzamendia
Tags: Time Series Forecasting, Machine Learning

Time Series Forecasting — A Getting Started Guide

Intro

When I started writing this post I thought of just explaining how to do predictions with a “simple” time series (aka univariate time series). The challenging part of the project I was in, however, was the fact that the prediction needed to be made in conjunction with multiple variables. For this reason, I decided to bring this guide a little bit closer to reality and use a multivariate time series.

Let’s get some concepts straight first…

A Multivariate TS is a time series with more than one time-dependent variable. Each variable depends on its past values but also has some dependency on other variables. This dependency is taken into account when predicting values. These variables can be endogenous or exogenous. I will be focusing on exogenous variables here.

An exogenous variable is one whose value is determined outside the model and is imposed on the model. In other words, variables that affect a model without being affected by it. Read more about exogenous variables here.

Many models can be used to solve a task like this, but SARIMAX is the one we’ll be working with. SARIMAX stands for Seasonal AutoRegressive Integrated Moving Average with eXogenous regressors.

All good! Now we will go through the steps one can follow to build a sales forecaster.