Python – Find text using beautifulSoup then replace in original soup variable

29 Nov, 2021
Python provides a library called BeautifulSoap to easily allow web scraping. BeautifulSoup object is provided by Beautiful Soup which is a web scraping framework for Python. Web scraping is the process of extracting data from the website using automated tools to make the process faster. The BeautifulSoup object represents the parsed document as a whole. In this article, we’ll be scrapping a simple website and replacing the content in the parsed “soup” variable.
For the purpose of this article, let’s create a virtual environment (venv) as it helps us to manage separate package installations for different projects and to avoid messing up with dependencies and interpreters!
More about, how to create a virtual environment can be read from here: Create a virtual environment
Navigate to your project directory and run this command to create a virtual environment named “env” in your project directory. 
python3 -m venv env
Activate the “env” by typing.
 source env/bin/activate
Having interpreter activated, we can see the name of an interpreter in our command line before :~$ symbol
BeautifulSoup: A library to scrape the web pages.
pip install bs4
requests: This makes the process of sending HTTP requests.
pip install requests
Let’s start by importing libraries and storing “GET” requests response in a variable.
Python3
import bs4from bs4 import BeautifulSoupimport requests # sending a GET req.response = requests.get("https://isitchristmas.today/")print(response.status_code)
Output:
200
A status of 200 implies a successful request.
Now let’s parse the content as a BeautifulSoup object to extract the title and header tags of the website (as for this article) and to replace it in the original soup variable. The find() method returns the first matching case from the soup object.
Python3
# create objectsoup = BeautifulSoup(r.text, "html.parser") # find titletitle = soup.find("title") # find headingheading = soup.find("h1") print(title)
Output:
Replacing the content of the parsed soup obj with the “.string” method. 
Python3
# replacetitle.string = "Is GFG day today?"heading.string = "Welcome to GFG"
Output:
Thus, the title tag and heading tags have been replaced in the original soup variable.
Note: We can’t push the modified page back to the website as those pages are rendered from servers where they are hosted.
Below is the complete program:
Python3
import bs4from bs4 import BeautifulSoupimport requests  # sending a GET requestsresponse = requests.get("https://isitchristmas.today/") # a status 200 implies a successful requests#print(response.status_code) soup = BeautifulSoup(response.text, "html.parser")#print(soup) title = soup.find("title")heading = soup.find("h1") # replacdetitle.string = "Is GFG day today?"heading.string = "Welcome to GFG" # display replaced contentprint(soup)# The title and the heading tag contents# get changed in the parsed soup obj.
Output:
 
surindertarika1234
kapoorsagar226
Picked
Python BeautifulSoup
Python bs4-Exercises
Technical Scripter 2020
Python
Technical Scripter
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
How to Install PIP on Windows ?
Python Classes and Objects
Python OOPs Concepts
Introduction To PYTHON
How to drop one or multiple columns in Pandas Dataframe
Python | os.path.join() method
Check if element exists in list in Python
How To Convert Python Dictionary To JSON?
Python | Get unique values from a list
Create a directory in Python