Fables of Data Science — Anscombe’s quartet | by dearC | Towards Data Science

..in a land far far away there used to live a man called Francis John “Frank” Anscombe. He was a statistician of great repute.
He wandered far and wide meeting many fellow practitioners and sat in the Citadel training young apprentices in the sacred art of statistics. The more he talked and trained people he saw a dangerous trend. People tended to ignore visualizations in favor of summary statistics. Everyone said it was too much effort to plot the data.
This was a dangerous, Frank was very worried. This plague was spreading was spreading fast, he did not have the time counter it. He had to do something, something that would help stop it.
So he rushed to talk to the council of olders but every one just laughed at him.
“When you have the data’s mean,What else do you want to examine?Standard deviation is just sublime,Plotting is a big waste of time!” — Master PoorPoetrix
Frank had never anticipated that the Council too was poisoned, he had to open their eyes.
So he traveled, traveled northwards into the Lands of Always Winter, to the peak of the Frostfang mountains. There he sat, deep in meditation. And after 5 minutes he realized that it is shit cold out in the mountains and he should have packed more woolen clothes.
Shivering in cold he went around looking for a cave to settle in and as luck would have it he found one.
There Frank meditated and he mediated deep, though some claimed he just fell asleep.
In his dreams he saw numbers, numbers and more numbers and it was then that he realized that the numbers were the key to his solution. Day after day he pondered on those numbers and finally he solved their mystery.
He called a bird and through him he sent to the Council a parchment with 4 sets of 11 data-points and requested the council as his last wish to plot those points:
.
.
+-------+--------+-------+-------+-------+-------+-------+------+|       I        |       II      |      III      |      IV      |+-------+--------+-------+-------+-------+-------+-------+------+| x     | y      | x     | y     | x     | y     | x     | y    | +-------+--------+-------+-------+-------+-------+-------+------+| 10.0  | 8.04   | 10.0  | 9.14  | 10.0  | 7.46  | 8.0   | 6.58 || 8.0   | 6.95   | 8.0   | 8.14  | 8.0   | 6.77  | 8.0   | 5.76 || 13.0  | 7.58   | 13.0  | 8.74  | 13.0  | 12.74 | 8.0   | 7.71 || 9.0   | 8.81   | 9.0   | 8.77  | 9.0   | 7.11  | 8.0   | 8.84 || 11.0  | 8.33   | 11.0  | 9.26  | 11.0  | 7.81  | 8.0   | 8.47 || 14.0  | 9.96   | 14.0  | 8.10  | 14.0  | 8.84  | 8.0   | 7.04 || 6.0   | 7.24   | 6.0   | 6.13  | 6.0   | 6.08  | 8.0   | 5.25 || 4.0   | 4.26   | 4.0   | 3.10  | 4.0   | 5.39  | 19.0  |12.50 || 12.0  | 10.84  | 12.0  | 9.13  | 12.0  | 8.15  | 8.0   | 5.56 || 7.0   | 4.82   | 7.0   | 7.26  | 7.0   | 6.42  | 8.0   | 7.91 || 5.0   | 5.68   | 5.0   | 4.74  | 5.0   | 5.73  | 8.0   | 6.89 |+-------+--------+-------+-------+-------+-------+-------+------+
As was their habit the council analyzed them using only descriptive statistics they got exactly similar results, they wondered what the old fool Frank was upto again:
                       Summary+-----+---------+-------+---------+-------+----------+| Set | mean(X) | sd(X) | mean(Y) | sd(Y) | cor(X,Y) |+-----+---------+-------+---------+-------+----------+|   1 |       9 |  3.32 |     7.5 |  2.03 |    0.816 ||   2 |       9 |  3.32 |     7.5 |  2.03 |    0.816 ||   3 |       9 |  3.32 |     7.5 |  2.03 |    0.816 ||   4 |       9 |  3.32 |     7.5 |  2.03 |    0.817 |+-----+---------+-------+---------+-------+----------+
But Master PoorPoetrix decided to honor his friend’s request and it was then that magic appeared when he plotted the data sets:
He saw that the graphs were completely different even though the summary was exactly similar.
The first scatter plot (top left) appeared to be a simple linear relationship, corresponding to two variables correlated and following the assumption of normality.
The second graph (top right) was not distributed normally; while a relationship between the two variables is obvious, it is not linear, and the Pearson correlation coefficient is not relevant. A more general regression and the corresponding coefficient of determination would be more appropriate.
In the third graph (bottom left), the distribution is linear, but should have a different regression line (a robust regression would have been called for). The calculated regression is offset by the one outlier which exerts enough influence to lower the correlation coefficient from 1 to 0.816.
Finally, the fourth graph (bottom right) shows an example when one high-leverage point is enough to produce a high correlation coefficient, even though the other data points do not indicate any relationship between the variables.
Master PoorPoetrix realized the folly of the Council’s ways and rectified them, this data-set came to be known as Anscombe’s quartet.
Frank had saved the world and lived out his last few days in the mountain cave as a happy old man.
.
.
Master PoorPoetrix is widely quoted to have summarized the incident with these immortal words:
To have analysis that you can trust,Plot your data, plot you must. — Master PoorPoetrix
https://en.wikipedia.org/wiki/Anscombe%27s_quartethttps://www.r-bloggers.com/using-and-abusing-data-visualization-anscombes-quartet-and-cheating-bonferroni/GIFs taken from https://giphy.com/
https://en.wikipedia.org/wiki/Anscombe%27s_quartet
https://www.r-bloggers.com/using-and-abusing-data-visualization-anscombes-quartet-and-cheating-bonferroni/
GIFs taken from https://giphy.com/