Extract Data From JustDial using Selenium - GeeksforGeeks

24 Oct, 2021
Let us see how to extract data from Justdial using Selenium and Python. Justdial is a company that provides local search for different services in India over the phone, website and mobile apps. In this article we will be extracting the following data:
Phone number
Name
Address
We can then save the data in a CSV file.
Approach:
Import the following modules: webdriver from selenium, ChromeDriverManager, pandas, time and os.Use the driver.get() method and pass the link you want to get information from.Use the driver.find_elements_by_class_name() method and pass ‘store-details’.Instantiate empty lists to store the values.Iterate the StoreDetails and start fetching the individual details that are required.Create a user-defined function strings_to_number() to convert the extracted string to numbers.Display the details and save them as a CSV file according to the requirements.
Import the following modules: webdriver from selenium, ChromeDriverManager, pandas, time and os.
Use the driver.get() method and pass the link you want to get information from.
Use the driver.find_elements_by_class_name() method and pass ‘store-details’.
Instantiate empty lists to store the values.
Iterate the StoreDetails and start fetching the individual details that are required.
Create a user-defined function strings_to_number() to convert the extracted string to numbers.
Display the details and save them as a CSV file according to the requirements.
Python3
# importing the modulesfrom selenium import webdriverfrom webdriver_manager.chrome import ChromeDriverManagerdriver = webdriver.Chrome(ChromeDriverManager().install())import pandas as pdimport timeimport os # driver.get method() will navigate to a page given by the URL addressdriver.get("https://www.justdial.com/Delhi/Ceiling-Tile-Dealers-Armstrong/nct-11271379") # the user-defined functiondef strings_to_num(argument):         switcher = {        'dc': '+',        'fe': '(',        'hg': ')',        'ba': '-',        'acb': '0',        'yz': '1',        'wx': '2',        'vu': '3',        'ts': '4',        'rq': '5',        'po': '6',        'nm': '7',        'lk': '8',        'ji': '9'    }    return switcher.get(argument, "nothing") # fetching all the store detailsstoreDetails = driver.find_elements_by_class_name('store-details') # instantiating empty listsnameList = []addressList = []numbersList = [] # iterating the storeDetailsfor i in range(len(storeDetails)):         # fetching the name, address and contact for each entry    name = storeDetails[i].find_element_by_class_name('lng_cont_name').text    address = storeDetails[i].find_element_by_class_name('cont_sw_addr').text    contactList = storeDetails[i].find_elements_by_class_name('mobilesv')         myList = []         for j in range(len(contactList)):                 myString = contactList[j].get_attribute('class').split("-")[1]             myList.append(strings_to_num(myString))     nameList.append(name)    addressList.append(address)    numbersList.append("".join(myList))     # initialize data of lists.data = {'Company Name': nameList,        'Address': addressList,        'Phone': numbersList} # Create DataFramedf = pd.DataFrame(data)print(df) # Save Data as .csvdf.to_csv('demo1.csv', mode = 'a', header = False)
  
Output:
 
 
anikakapoor
anikaseth98
Python Selenium-Exercises
Python-selenium
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Comments
Old Comments
How to Install PIP on Windows ?
How to drop one or multiple columns in Pandas Dataframe
Python OOPs Concepts
Python | Get unique values from a list
Check if element exists in list in Python
Python Classes and Objects
Python | os.path.join() method
How To Convert Python Dictionary To JSON?
Python | Pandas dataframe.groupby()
Create a directory in Python