How To Extract Data From Common File Formats in Python? - GeeksforGeeks

13 Jan, 2021
Sometimes work with some datasets must have mostly worked with .csv(Comma Separated Value) files only. They are really a great starting point in applying Data Science techniques and algorithms. But many of us will land up in Data Science firms or take up real-world projects in Data Science sooner or later. Unfortunately in real-world projects, the data won’t be available to us in a neat .csv file. There we have to extract data from different sources like images, pdf files, doc files, image files, etc. In this article, we will see the perfect start to tackle those situations.
Below we will see how to extract relevant information from multiple such sources.
Note that if the Excel file has a single sheet then the same method to read CSV file (pd.read_csv(‘File.xlsx’)) might work. But it won’t in the case of multiple sheet files as shown in the below image where there are 3 sheets( Sheet1, Sheet2, Sheet3). In this case, it will just return the first sheet.
Excel sheet used: Click Here.
Example: We will see how to read this excel-file.
Python3
# import Pandas libraryimport pandas as pd  # Read our file. Here sheet_name=1# means we are reading the 2nd sheet or Sheet2df = pd.read_excel('Sample1.xlsx', sheet_name = 1)df.head()
Output:
Now let’s read a selected column of the same sheet:
Python3
# Read only column A, B, C of all# the four columns A,B,C,D in Sheet2df=pd.read_excel('Sample1.xlsx',                 sheet_name = 1, usecols = 'A : C')df.head()
Output:
Now let’s read all sheet together:
Sheet1 contains columns A, B, C; Sheet2 contains A, B, C, D and Sheet3 contains B, D. We will see a simple example below on how to read all the 3 sheets together and merge them into common columns.
Python3
df2 = pd.DataFrame()for i in df.keys():    df2 = pd.concat([df2, df[i]],                     axis = 0)  display(df2)
Output:
Now we will discuss how to extract text from images.
For enabling our python program to have Character recognition capabilities, we would be making use of pytesseract OCR library. The library could be installed onto our python environment by executing the following command in the command interpreter of the OS:-
pip install pytesseract
The library (if used on Windows OS) requires the tesseract.exe binary to be also present for proper installation of the library. During the installation of the aforementioned executable, we would be prompted to specify a path for it. This path needs to be remembered as it would be utilized later on in the code. For most installations the path would be C:\\Program Files (x86)\\Tesseract-OCR\\tesseract.exe. 
Image for demonstration:
Python3
# We import necessary libraries. # The PIL Library is used to read the imagesfrom PIL import Imageimport pytesseract  # Read the imageimage = Image.open(r'pic.png')  # Perform the information extraction from images# Note below, put the address where tesseract.exe # file is located in your systempytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe'  print(pytesseract.image_to_string(image))
Output:
GeeksforGeeks
Here we will extract text from the doc file using docx module.
For installation:
pip install python-docx
Image for demonstration:  Aniket_Doc.docx 
Example 1: First we’ll extract the title:
Python3
# Importing our library and reading the doc fileimport docxdoc = docx.Document('csv/g.docx')  # Printing the titleprint(doc.paragraphs[0].text)
Output:
My Name Aniket
Example 2: Then we’ll extract the different texts present(excluding the table).
Python3
# Getting all the text in the doc filel=[doc.paragraphs[i].text for i in range(len(doc.paragraphs))]  # There might be many useless empty# strings present so removing theml=[i for i in l if len(i)!=0]print(l)
Output:
[‘My Name Aniket’, ‘               Hello I am Aniket’, ‘I am giving tutorial on how to extract text from MS Doc.’, ‘Please go through it carefully.’]
Example 3: Now we’ll extract the table:
Python3
# Since there are only one table in# our doc file we are using 0. For multiple tables# you can use suitable for tooptable = doc.tables[0]  # Initializing some empty listlist1 = []list2 = []  # Looping through each row of tablefor i in range(len(table.rows)):        # Looping through each column of a row    for j in range(len(table.columns)):          # Extracting the required text        list1.append(table.rows[i].cells[j].paragraphs[0].text)      list2.append(list1[:])    list1.clear()  print(list2)
Output:
[['A', 'B', 'C'], ['12', 'aNIKET', '@@@'], ['3', 'SOM', '+12&']]
The task is to extract Data( Image, text) from PDF in Python. We will extract the images from PDF files and save them using PyMuPDF library. First, we would have to install the PyMuPDF library using Pillow.
pip install PyMuPDF Pillow
Example 1:
Now we will extract data from the pdf version of the same doc file.
Python3
# import moduleimport fitz  # Reading our pdf filedocu=fitz.open('file.pdf')  # Initializing an empty list where we will put all texttext_list=[]  # Looping through all pages of the pdf filefor i in range(docu.pageCount):       # Loading each page  pg=docu.loadPage(i)      # Extracting text from each page  pg_txt=pg.getText('text')      # Appending text to the empty list  text_list.append(pg_txt)    # Cleaning the text by removing useless# empty strings and unicode character '\u200b'text_list=[i.replace(u'\u200b','') for i in text_list[0].split('\n') if len(i.strip()) ! = 0]print(text_list)
Output:
[‘My Name Aniket ‘, ‘               Hello I am Aniket ‘, ‘I am giving tutorial on how to extract text from MS Doc. ‘, ‘Please go through it carefully. ‘, ‘A ‘, ‘B ‘, ‘C ‘, ’12 ‘, ‘aNIKET ‘, ‘@@@ ‘, ‘3 ‘, ‘SOM ‘, ‘+12& ‘]
Example 2: Extract image from PDF.
Python3
# Iterating through the pagesfor current_page in range(len(docu)):      # Getting the images in that page  for image in docu.getPageImageList(current_page):          # get the XREF of the image . XREF can be thought of a    # container holding the location of the image    xref=image[0]          # extract the object i.e,    # the image in our pdf file at that XREF    pix=fitz.Pixmap(docu,xref)          # Storing the image as .png    pix.writePNG('page %s - %s.png'%(current_page,xref))
The image is stored in our current file location as in format page_no.-xref.png. In our case, its name is page 0-7.png.
Now let’s plot view the image.
Python3
# Import necessary libraryimport matplotlib.pyplot as plt  # Read and display the imageimg=plt.imread('page 0 - 7.png')plt.imshow(img)
Output:
python-utility
Technical Scripter 2020
Python
Technical Scripter
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
How to Install PIP on Windows ?
Check if element exists in list in Python
How To Convert Python Dictionary To JSON?
Python Classes and Objects
How to drop one or multiple columns in Pandas Dataframe
Python | Get unique values from a list
Python | os.path.join() method
Defaultdict in Python
Create a directory in Python
Python | Pandas dataframe.groupby()