Data analysis and Visualization with Python - GeeksforGeeks

21 Feb, 2018
Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages, and makes importing and analyzing data much easier. In this article, I have used Pandas to analyze data on Country Data.csv file from UN public Data Sets of a popular ‘statweb.stanford.edu’ website.As I have analyzed the Indian Country Data, I have introduced Pandas key concepts as below. Before going through this article, have a rough idea of basics from matplotlib and csv.
InstallationEasiest way to install pandas is to use pip:
pip install pandas
or, Download it from here
Creating A DataFrame in Pandas
Creation of dataframe is done by passing multiple Series into the DataFrame class using pd.Series method. Here, it is passed in the two Series objects, s1 as the first row, and s2 as the second row.Example:
# assigning two series to s1 and s2s1 = pd.Series([1,2])s2 = pd.Series(["Ashish", "Sid"])# framing series objects into datadf = pd.DataFrame([s1,s2])# show the data framedf  # data framing in another way# taking index and column valuesdframe = pd.DataFrame([[1,2],["Ashish", "Sid"]],        index=["r1", "r2"],        columns=["c1", "c2"])dframe  # framing in another way # dict-like containerdframe = pd.DataFrame({        "c1": [1, "Ashish"],        "c2": [2, "Sid"]})dframe
Output:
    

Importing Data with Pandas
The first step is to read the data. The data is stored as a comma-separated values, or csv, file, where each row is separated by a new line, and each column by a comma (,). In order to be able to work with the data in Python, it is needed to read the csv file into a Pandas DataFrame. A DataFrame is a way to represent and work with tabular data. Tabular data has rows and columns, just like this csv file(Click Download).Example:
# Import the pandas library, renamed as pdimport pandas as pd  # Read IND_data.csv into a DataFrame, assigned to dfdf = pd.read_csv("IND_data.csv")  # Prints the first 5 rows of a DataFrame as defaultdf.head()  # Prints no. of rows and columns of a DataFramedf.shape
Output:


29,10

Indexing DataFrames with Pandas
Indexing can be possible using the pandas.DataFrame.iloc method. The iloc method allows to retrieve as  many as rows and columns by position.Examples:
# prints first 5 rows and every column which replicates df.head()df.iloc[0:5,:]# prints entire rows and columnsdf.iloc[:,:]# prints from 5th rows and first 5 columnsdf.iloc[5:,:5]
Indexing Using Labels in Pandas
Indexing can be worked with labels using the pandas.DataFrame.loc method, which allows to index using labels instead of positions.Examples:
# prints first five rows including 5th index and every columns of dfdf.loc[0:5,:]# prints from 5th rows onwards and entire columnsdf = df.loc[5:,:]
The above doesn’t actually look much different from df.iloc[0:5,:]. This is because while row labels can take on any values, our row labels match the positions exactly. But column labels can make things much easier when working with data. Example:
# Prints the first 5 rows of Time period# value df.loc[:5,"Time period"]


DataFrame Math with Pandas
Computation of data frames can be done by using Statistical Functions of pandas tools.Examples:
# computes various summary statistics, excluding NaN valuesdf.describe()# for computing correlationsdf.corr()# computes numerical data ranksdf.rank()


 



Pandas Plotting
Plots in these examples are made using standard convention for referencing the matplotlib API which provides the basics in pandas to easily create decent looking plots.Examples:
# import the required module import matplotlib.pyplot as plt# plot a histogram df['Observation Value'].hist(bins=10)  # shows presence of a lot of outliers/extreme valuesdf.boxplot(column='Observation Value', by = 'Time period')  # plotting points as a scatter plotx = df["Observation Value"]y = df["Time period"]plt.scatter(x, y, label= "stars", color= "m",             marker= "*", s=30)# x-axis labelplt.xlabel('Observation Value')# frequency labelplt.ylabel('Time period')# function to show the plotplt.show()




Data Analysis and Visualization with Python | Set 2
Reference:
http://pandas.pydata.org/pandas-docs/stable/tutorials.html
https://www.datacamp.com
This article is contributed by Afzal_Saan. If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Python Dictionary
Read a file line by line in Python
Enumerate() in Python
How to Install PIP on Windows ?
Iterate over a list in Python
Different ways to create Pandas Dataframe
Python String | replace()
Python program to convert a list to string
Create a Pandas DataFrame from Lists
Reading and Writing to text files in Python