How to parse local HTML file in Python? - GeeksforGeeks

16 Mar, 2021
Prerequisites: Beautifulsoup
Parsing means dividing a file or input into pieces of information/data that can be stored for our personal use in the future. Sometimes, we need data from an existing file stored on our computers, parsing technique can be used in such cases. The parsing includes multiple techniques used to extract data from a file. The following includes Modifying the file, Removing something from the file, Printing data, using the recursive child generator method to traverse data from the file, finding the children of tags, web scraping from a link to extract useful information, etc.
Using the prettify method to modify the HTML code from- https://festive-knuth-1279a2.netlify.app/, look better. Prettify makes the code look in the standard form like the one used in VS Code.
Example:
Python3
# Importing BeautifulSoup class from the bs4 modulefrom bs4 import BeautifulSoup  # Importing the HTTP libraryimport requests as req  # Requesting for the websiteWeb = req.get('https://festive-knuth-1279a2.netlify.app/')  # Creating a BeautifulSoup object and specifying the parserS = BeautifulSoup(Web.text, 'lxml')  # Using the prettify methodprint(S.prettify())
Output:
A tag can be removed by using the decompose method and the select_one method with the CSS selectors to select and then remove the second element from the li tag and then using the prettify method to modify the HTML code from the index.html file.
Example:
File Used:
Python3
# Importing BeautifulSoup class from the bs4 modulefrom bs4 import BeautifulSoup  # Opening the html fileHTMLFile = open("index.html", "r")  # Reading the fileindex = HTMLFile.read()  # Creating a BeautifulSoup object and specifying the parserS = BeautifulSoup(index, 'lxml')  # Using the select-one method to find the second element from the li tagTag = S.select_one('li:nth-of-type(2)')  # Using the decompose methodTag.decompose()  # Using the prettify method to modify the codeprint(S.body.prettify())
Output:
Tags can be found normally and printed normally using print().
Example:
Python3
# Importing BeautifulSoup class from the bs4 modulefrom bs4 import BeautifulSoup  # Opening the html fileHTMLFile = open("index.html", "r")  # Reading the fileindex = HTMLFile.read()  # Creating a BeautifulSoup object and specifying the parserParse = BeautifulSoup(index, 'lxml')  # Printing html code of some tagsprint(Parse.head)print(Parse.h1)print(Parse.h2)print(Parse.h3)print(Parse.li)
Output:
The recursiveChildGenerator method is used to traverse tags, which recursively finds all the tags within tags from the file.
Example:
Python3
# Importing BeautifulSoup class from the bs4 modulefrom bs4 import BeautifulSoup  # Opening the html fileHTMLFile = open("index.html", "r")  # Reading the fileindex = HTMLFile.read()  # Creating a BeautifulSoup object and specifying the parserS = BeautifulSoup(index, 'lxml')  # Using the recursiveChildGenerator method to traverse the html filefor TraverseTags in S.recursiveChildGenerator():  # Traversing the names of the tags    if TraverseTags.name:      # Printing the names of the tags        print(TraverseTags.name)
Output:
Using the name attribute of the tag to print its name and the text attribute to print its text along with the code of the tag- ul from the file.
Example:
Python3
# Importing BeautifulSoup class from the bs4 modulefrom bs4 import BeautifulSoup  # Opening the html fileHTMLFile = open("index.html", "r")  # Reading the fileindex = HTMLFile.read()  # Creating a BeautifulSoup object and specifying the parserS = BeautifulSoup(index, 'lxml')  # Printing the Code, name, and text of a tagprint(f'HTML: {S.ul}, name: {S.ul.name}, text: {S.ul.text}')
Output:
The Children attribute is used to get the children of a tag. The Children attribute returns ‘tags with spaces’ between them, we’re adding a condition- e. name is not None to print only names of the tags from the file.
Example:
Python3
# Importing BeautifulSoup class from the bs4 modulefrom bs4 import BeautifulSoup  # Opening the html fileHTMLFile = open("index.html", "r")  # Reading the fileindex = HTMLFile.read()  # Creating a BeautifulSoup object and specifying the parserS = BeautifulSoup(index, 'lxml')  # Providing the sourceAttr = S.html  # Using the Children attribute to get the children of a tag# Only contain tag names and not the spacesAttr_Tag = [e.name for e in Attr.children if e.name is not None]  # Printing the childrenprint(Attr_Tag)
Output:
Finding Children at all levels of a tag:
The Descendants attribute is used to get all the descendants (Children at all levels) of a tag from the file.
Example:
Python3
# Importing BeautifulSoup class from the bs4 modulefrom bs4 import BeautifulSoup  # Opening the html fileHTMLFile = open("index.html", "r")  # Reading the fileindex = HTMLFile.read()  # Creating a BeautifulSoup object and specifying the parserS = BeautifulSoup(index, 'lxml')  # Providing the sourceDes = S.body  # Using the descendants attributeAttr_Tag = [e.name for e in Des.descendants if e.name is not None]  # Printing the childrenprint(Attr_Tag)
Output:
Using find_all():
The find_all method is used to find all the elements (name and text) inside the p tag from the file.
Example:
Python3
# Importing BeautifulSoup class from the bs4 modulefrom bs4 import BeautifulSoup  # Opening the html fileHTMLFile = open("index.html", "r")  # Reading the fileindex = HTMLFile.read()  # Creating a BeautifulSoup object and specifying the parserS = BeautifulSoup(index, 'lxml')  # Using the find_all method to find all elements of a tagfor tag in S.find_all('p'):    # Printing the name, and text of p tag    print(f'{tag.name}: {tag.text}')
Output:
CSS selectors to find elements: 
Using the select method to use the CSS selectors to find the second element from the li tag from the file.
Example:
Python3
# Importing BeautifulSoup class from the bs4 modulefrom bs4 import BeautifulSoup  # Opening the html fileHTMLFile = open("index.html", "r")  # Reading the fileindex = HTMLFile.read()  # Creating a BeautifulSoup object and specifying the parserS = BeautifulSoup(index, 'lxml')  # Using the select method# Prints the second element from the li tagprint(S.select('li:nth-of-type(2)'))
Output:
Picked
Python BeautifulSoup
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
How to Install PIP on Windows ?
Check if element exists in list in Python
How To Convert Python Dictionary To JSON?
How to drop one or multiple columns in Pandas Dataframe
Python Classes and Objects
Python | os.path.join() method
Python | Get unique values from a list
Create a directory in Python
Defaultdict in Python
Python | Pandas dataframe.groupby()