What is Google API Vision? And how to use it | by Christophe Pere | Towards Data Science

This post finds his root in an interesting project of knowledge extraction. The first step was to extract the text of pdf documents. The company that I work for is based on the Google platform, so naturally, I would like to use the OCR of the API Vision but, can’t find an easy way to use the API to extract text. So here this post.
The notebook of this post is available on GitHub
Google released the API to help people, industry, and researchers to use their functionalities.
Google Cloud's Vision API has powerful machine learning models pre-trained through REST and RPC APIs. Tag images and quickly organize them into millions of predefined categories. You will be able to detect objects and faces, read printed or handwritten text, and integrate useful metadata into your image catalog. (source: API Vision)
The part of the API that interested us for this post is the OCR part.
Optical Character Recognition or OCR is a technology where characters are recognized and detected inside an image. Most of the time Convolutional Neural Networks (CNN) are trained on a very large dataset of characters and numbers in different types and colors. You can imagine a small window slicing on each pixel or group of pixels to detect characters or partial characters, spaces, forms, lines etc.
A service account is a special type of Google account intended to represent a non-human user that needs to authenticate and be authorized to access data in Google APIs. (source: IAM google cloud)
Basically you can imagine it as an RSA key (encrypted key to communicate with high security between machine via the internet) with which you can connect to Google services (API, GCS, IAM...). Its basic form is a json file.
Here, I will show you the different functions to use the API and extract the text from the image automatically.
Libraries needed to be installed:
!pip install google-cloud!pip install google-cloud-storage!pip install google-cloud-pubsub!pip install google-cloud-vision!pip install pdf2image!pip install google-api-python-client!pip install google-auth
The libraries used:
from pdf2image import convert_from_bytesimport globfrom tqdm import tqdmimport base64import jsonimport osfrom io import BytesIOimport numpy as npimport iofrom PIL import Imagefrom google.cloud import pubsub_v1from google.cloud import visionfrom google.oauth2 import service_accountimport googleapiclient.discovery# to see a progress bartqdm().pandas()
The OCR can take pdf, tiff and jpeg formats to be used in the API.In this post we will convert the pdf into jpeg to concatenate many pages in one picture. Two manners of using jpeg:
First, you could convert your pdf in jpeg files and save them into another repository:
# Name files where the pdf are and where you want to save the resultsNAME_INPUT_FOLDER = "PDF FOLDER NAME"NAME_OUTPUT_FOLDER= "RESULT TEXTS FOLDER"list_pdf = glob.glob(NAME_INPUT_FOLDER+"/*.pdf") # stock the name of the pdf files # Loop over all the filesfor i in list_pdf:        # convert the pdf into jpeg        pages = convert_from_path(i, 500)                for page in tqdm(enumerate(pages)):            # save each page into jpeg             page[1].save(NAME_OUTPUT_FOLDER+"/"+i.split('/')[-1].split('.')[0]+'_'+str(page[0])+'.jpg', 'JPEG') # keep the name of the document and add increment 
Here, you can use your jpeg document with the API. But, you can do it better without saving the jpeg file and use it in memory to call the API directly.
Before going deeper we need to configure the credentials of the Vision API. You’ll see, it’s very simple:
SCOPES = ['https://www.googleapis.com/auth/cloud-vision']SERVICE_ACCOUNT_FILE = "PUT the PATH of YOUR SERVICE ACCOUNT JSON FILE HERE"# Configure the google credentialscredentials = service_account.Credentials.from_service_account_file(        SERVICE_ACCOUNT_FILE, scopes=SCOPES)
This needed more code because we also concatenate 10 pages of documents to create a “big picture” and feed it to the API. One call versus 10 is better for the price because you’ll pay each time you’ll request the API.
Let’s go:
With these two functions, you’ll be able to load a pdf file, convert it into bytes, create a “big picture” and feed it to the function detect_text_document() (details below).
The function detect_text_document takes in input the content of the pictures and the credentials (information of your service account).
The output is a text extracted from the images. The goal of this function is to concatenate words into paragraphs and documents.
You can use this block of functions like this:
for doc_pdf in tqdm(list_pdf):        # call the function which convert into jpeg, stack 10 images        # and call the API, save the output into txt file         concat_file_ocr(doc_pdf)
The input is just the path obtained with the glob function. The credentials were generated in the Setup Credentials part. This loop will take each pdf of the input files, call the API with a jpeg file obtained by converting the pdf and save text files containing the detection.
Here, you reach the end of the tutorial on how to use the Vision API and generate text files containing the detection automatically. You know how to configure credentials with your service account and convert a pdf into a jpeg file (one jpeg per page). Is it all? No, I have some bonuses for you (see below).
The previous functions allow you to use the API with the concatenation of pages. But, we can use the API per page of the pdf document. The function below will request the API for each page of the convert pdf into a jpeg format.
It’s very easy to use it, just call this function with the path of the pdf folder and the credentials. Like this:
if per_page: # option True if you want to use per page    # call the API vision per page of the pdf    for i in tqdm(list_pdf):        # open the pdf and convert it into a PlImage format jpeg        call_ocr_save_txt(i, cred=credentials)
Just for fun, you can use this API with multiprocessing (ok it’s not real multiprocessing in python with the Global Interpreter Lock (GIL)). But, here the code:
if multi_proc:    nb_threads = mp.cpu_count() # return the number of CPU    print(f"The number of available CPU is {nb_threads}")        # if you want to use the API without stacking the pages    if per_page:        # create threads corresponding to the number specified        pool = mp.Pool(processes=nb_threads)            # map the function with part of the list for each thread        result = pool.map(call_ocr_save_txt, list_pdf)             if per_document:        pool = mp.Pool(processes=nb_threads)         result = pool.map(concat_file_ocr, list_pdf)