BeautifulSoup - Modifying the tree - GeeksforGeeks

07 Sep, 2021
Prerequisites: BeautifulSoup
Beautifulsoup is a Python library used for web scraping. This powerful python tool can also be used to modify html webpages. This article depicts how beautifulsoup can be employed to modify the parse tree. BeautifulSoup is used to search the parse tree and allow you to modify the tree. You can rename tag, change the values of its attributes, add and delete attribute.
You can change the name of the tag and modify its attribute by adding or deleting them.
To change tag name:
Syntax: tag.name = “new_tag”
To modify its attribute or to add new attribute:
Syntax: tag[“attribute”] = “value”
To delete any attribute:
Syntax: del tag[“attribute”]
A tree can also be modified by inserting new elements at required places.
insert() function will insert new element at any position
Syntax: tag.insert()
insert_after() function will insert element after something in the parse tree.
Syntax: tag.insert_after()
insert_before() function will insert element before something in the parse tree.
Syntax: tag.insert_before()
Approach :
Import module
Scrap data from webpage
Parse the string scraped to html
Select tag within which modification has to be performed
Make required changes
Example 1: 
Python3
# importing modulefrom bs4 import BeautifulSoup markup = """<p class="para">gfg</p>    """ # parsering string to HTMLsoup = BeautifulSoup(markup, 'html.parser') # extracting a tagtag = soup.p print("Before modifying the tag name: ")print(tag)print() # modifying tag nametag.name = "div" print("After modifying the tag name: ")print(tag)print()# modifying its class attributetag['class'] = "div_class" # adding new attributetag['id'] = "div_id" print("After modifying and adding attributes: ")print(tag)print() # to delete any attributesdel tag["class"] print("After deleting class attribute: ")print(tag)print() # modifying the tags contenttag.string = "Geeks" print("After modifying tag string: ")print(tag)print() # using insert function.tag = soup.divprint("Before inserting: ")print(tag)print() # inserting contenttag.insert(1, " for Geeks")print("After inserting: ")print(tag)print()
Output:
Example 2:
Python3
# importing modulefrom bs4 import BeautifulSoup soup = BeautifulSoup("<b>| A Computer Science portal</b>", 'html.parser') tag = soup.new_tag("p")tag.string = "Geeks"  # insert beforesoup.b.string.insert_before(tag)print(soup.b)print() # insert aftersoup.b.p.insert_after(soup.new_string(" for Geeks"))print(soup.b)
Output:
The tree can be modified by adding a new tag at any required location. We can also wrap the element to modify it.
new_tag() function will add a new tag
Syntax: new_tag(“attribute”)
wrap() function will enclose an element in the tag you specify and returns a new wrapper
Syntax: wrap()
unwrap() function unwrap the wrapped elements.
Syntax: unwrap()
Example:
Python3
# importing modulefrom bs4 import BeautifulSoup markup = '    <p>Geeks for Geeks</p>    ' # parsering string to HTMLsoup = BeautifulSoup(markup, 'html.parser')print(soup) # wrapping around the stringsoup.p.string.wrap(soup.new_tag("i"))print(soup) # wrapping around the tagsoup.p.wrap(soup.new_tag("div"))print(soup) # unwrapping the i tag soup.p.i.unwrap() print(soup) old_tag = soup.div # new tagnew_tag = soup.new_tag('div')new_tag.string = "| A Computer Science portal for geeks" # adding new tagold_tag.append(new_tag) print(soup)
Output:
replace_with() function will replace old tag or string with new tag or string in the parse tree.
Syntax: replace_with()
Example:
Python3
# importing BeautifulSoup Modulefrom bs4 import BeautifulSoup markup = '<a href="http://gfg.com/">Geeks for Geeks <i>gfg.com</i></a>' # parsering string to HTMLsoup = BeautifulSoup(markup, 'html.parser') # tag to be replacedold_tag = soup.a # new tagnew_tag = soup.new_tag("p") # input stringnew_tag.string = "gfg.in" '''replacing tag  page_element.replace_with("string")removes a tag or string from the tree, and replacesit with the tag or string of your choice.''' old_tag.i.replace_with(new_tag) print(old_tag)
Output:
<a href=”http://gfg.com/”>Geeks for Geeks <p>gfg.in</p></a>
For adding new contents to an existing tag can be done by append() function or NavigableString() constructor.
Syntax: tag.append(“content”)
Example:
Python3
# importing modulefrom bs4 import BeautifulSoupfrom bs4 import NavigableString markup = """<a href="https://www.geeksforgeeks.org/">Geeks for Geeks</a>""" # parsering string to HTMLsoup = BeautifulSoup(markup, 'html.parser') # extracting a tagtag = soup.a # appending contenttag.append("| A Computer Science portal")print(tag) # appending content using navigableString constructornew_str = NavigableString(" for geeks")tag.append(new_str)print(tag)
Output:
<a href=”https://www.geeksforgeeks.org/”>Geeks for Geeks| A Computer Science portal</a>
<a href=”https://www.geeksforgeeks.org/”>Geeks for Geeks| A Computer Science portal for geeks</a>
A tree can be modified by removing content from it or by removing element also.
clear() removes the contents of the tag.
Syntax: clear()
extract() removes a tag or strings from the tree.
Syntax: extract()
decompose() removes the tag and delete it all content.
Syntax: decompose()
Example:
Python3
# importing modulefrom bs4 import BeautifulSoup markup = '<a href="https://www.geeksforgeeks.org/">Geeks for Geeks <i>| A Computer Science portal</i></a>' # parsering string to HTMLsoup = BeautifulSoup(markup, 'html.parser') tag = soup.aprint(tag)print() # clearing its all contenttag.clear()print(tag)print() # extracting i tag# parsering string to HTMLsoup2 = BeautifulSoup(markup, 'html.parser') a_tag = soup2.a print(a_tag)print()i_tag = soup2.i.extract() print(a_tag)print() # decomposing i tag# parsering string to HTMLsoup2 = BeautifulSoup(markup, 'html.parser') a_tag = soup2.a print(a_tag)print()i_tag = soup2.i.decompose() print(a_tag)
Output:
sweetyty
singghakshay
anikaseth98
kapoorsagar226
Picked
Python BeautifulSoup
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
How to Install PIP on Windows ?
Check if element exists in list in Python
How To Convert Python Dictionary To JSON?
How to drop one or multiple columns in Pandas Dataframe
Python Classes and Objects
Python | Get unique values from a list
Python | os.path.join() method
Create a directory in Python
Defaultdict in Python
Python | Pandas dataframe.groupby()