Build Fuel Price Tracker Using Python - GeeksforGeeks

31 Mar, 2022
In this modern-day lifestyle, fuel has become a necessity for all human beings. It is a base for our life-style. So, we are going to write a script to track their price using Python.
Modules Needed
bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come built-in with Python. To install this type the below command in the terminal.
pip install bs4
requests: Request allows you to send HTTP/1.1 requests extremely easily. This module also does not come built-in with Python. To install this type the below command in the terminal.
pip install requests
Let’s see the stepwise execution of the script
Step 1: Import all dependence
Python3
# import moduleimport pandas as pdimport requestsfrom bs4 import BeautifulSoup
Step 2: Create a URL get function
Python3
# user define function# Scrape the datadef getdata(url):    r = requests.get(url)    return r.text
Step 3: Now pass the URL into the getdata() function and Convert that data into HTML code
Python3
# link for extract html datahtmldata = getdata("https://www.goodreturns.in/petrol-price.html")soup = BeautifulSoup(htmldata, 'html.parser')result = soup.find_all("div", class_="gold_silver_table")print(result)
Output :
[<div class=”gold_silver_table”> <table border=”0′′ cellpadding=”1′′ cellspacing=”1′′ width=”100%”> <tr class=”first”> <td class=”heading” width=”200′′>City</td> <td class=”heading” width=”200′′>Today Price</td> <td class=”heading” width=”200′′>Yesterday’s Price</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-new-delhi.html” title=”New Delhi”>New Delhi</a></td> <td> ₹ 82.08</td> <td> ₹ 82.03</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-kolkata.html” title=”Kolkata”>Kolkata</a></td> <td> ₹ 83.57</td> <td> ₹ 83.52</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-mumbai.html” title=”Mumbai”>Mumbai</a></td> <td> ₹ 88.73</td> <td> ₹ 88.68</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-chennai.html” title=”Chennai”>Chennai</a></td> <td> ₹ 85.04</td> <td> ₹ 85.00</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-gurgaon.html” title=”Gurgaon”>Gurgaon</a></td> <td> ₹ 79.92</td> <td> ₹ 79.84</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-noida.html” title=”Noida”>Noida</a></td> <td> ₹ 82.23</td> <td> ₹ 82.30</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-bangalore.html” title=”Bangalore”>Bangalore</a></td> <td> ₹ 84.75</td> <td> ₹ 84.70</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-bhubaneswar.html” title=”Bhubaneswar”>Bhubaneswar</a></td> <td> ₹ 82.47</td> <td> ₹ 82.59</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-chandigarh.html” title=”Chandigarh”>Chandigarh</a></td> <td> ₹ 78.96</td> <td> ₹ 78.92</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-hyderabad.html” title=”Hyderabad”>Hyderabad</a></td> <td> ₹ 85.30</td> <td> ₹ 85.25</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-jaipur.html” title=”Jaipur”>Jaipur</a></td> <td> ₹ 90.08</td> <td> ₹ 89.24</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-lucknow.html” title=”Lucknow”>Lucknow</a></td> <td> ₹ 82.20</td> <td> ₹ 82.09</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-patna.html” title=”Patna”>Patna</a></td> <td> ₹ 84.73</td> <td> ₹ 84.88</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-trivandrum.html” title=”Trivandrum”>Trivandrum</a></td> <td> ₹ 83.91</td> <td> ₹ 84.03</td> </tr> </table> </div>]
Note: These scripts will give you only Raw data in String format you have to print your data with your needs. 
Step 4: Now, Search your needed data into sting with soup.find_all().
Python3
# Declare string var# Declare listmydatastr = ''result = [] # searching all tr in the html data# storing as a stringfor table in soup.find_all('tr'):    mydatastr += table.get_text() # set according to your requiredmydatastr = mydatastr[1:]itemlist = mydatastr.split("\n\n") for item in itemlist[:-5]:    result.append(item.split("\n")) result
Output :
Step 4: Make a DataFrame for displaying your result.
Python3
# Calling DataFrame constructor on listdf = pd.DataFrame(result[:-8])df
Complete code:
Python3
# import moduleimport requestsimport pandas as pdfrom bs4 import BeautifulSoup # link for extract html data  def getdata(url):    r = requests.get(url)    return r.text  htmldata = getdata("https://www.goodreturns.in/petrol-price.html")soup = BeautifulSoup(htmldata, 'html.parser') # Declare string var# Declare listmydatastr = ''result = [] # searching all tr in the html data# storing as a stringfor table in soup.find_all('tr'):    mydatastr += table.get_text() # set according to your requiredmydatastr = mydatastr[1:]itemlist = mydatastr.split("\n\n") for item in itemlist[:-5]:    result.append(item.split("\n")) # Calling DataFrame constructor on listdf = pd.DataFrame(result[:-8])df
Output :
simmytarika5
Python web-scraping-exercises
Python-pandas
Python-requests
python-utility
Web-scraping
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
How to Install PIP on Windows ?
How To Convert Python Dictionary To JSON?
Check if element exists in list in Python
How to drop one or multiple columns in Pandas Dataframe
Python Classes and Objects
Python | os.path.join() method
Create a directory in Python
Defaultdict in Python
Python | Pandas dataframe.groupby()
Python | Get unique values from a list