BeautifulSoup - Find all <li> in <ul> - GeeksforGeeks

15 Mar, 2021
Prerequisites: Beautifulsoup
Beautifulsoup is a Python module used for web scraping. In this article, we will discuss how contents of <li> tags can be retrieved from <ul> using Beautifulsoup. 
bs4: Beautiful Soup(bs4)  is a Python library for pulling data out of HTML and XML files.
requests: Requests allow you to send HTTP/1.1 requests extremely easily. This module also does not comes built-in with Python.
Import the modules
Provide an URL that has ul and li tags
Make the requests
Create the beautifulsoup object
Find the required tags
Retrieve the contents under li
Below the code, the HTML snippet contains a body with ul and li tags that have been obtained by the beautifulsoup object.
In this method, we use the descendants attribute present in beautifulsoup which basically returns a list iterator object having all the descendants/children of the parent tag, here parent is <ul> tag.
First, import the required modules, then provide the URL and create its requests object that will be parsed by the beautifulsoup object. Now with the help of find() function in beautifulsoup we will find the <body> and its corresponding <ul> tags. After this, the descendants attribute will give us the list iterator object which is needed to convert back into list. This list has a next line item, the tags with text, and finally the only text. So, we will print every second successive element of the list.
Example:
Python3
# importing the modulesimport requestsfrom bs4 import BeautifulSoup  # providing urlurl = "https://auth.geeksforgeeks.org/user/adityaprasad1308/articles"  # creating requests objecthtml = requests.get(url).content  # creating soup objectdata = BeautifulSoup(html, 'html.parser')  # finding parent <ul> tagparent = data.find("body").find("ul")  # finding all <li> tagstext = list(parent.descendants)  # printing the content in <li> tagprint(text)for i in range(2, len(text), 2):    print(text[i], end=" ")
Output:
Approach is same as the above example, but instead of finding the body we will find ul tags and then find all the li tags with the help of find_all() function which takes the tag name as an argument and returns all the li tags. After this we will simply iterate over all the <li> tags and with the help of text attribute we will print the text present in the <li> tag.
Example:
Python3
# importing the modulesimport requestsfrom bs4 import BeautifulSoup  # providing urlurl = 'https://auth.geeksforgeeks.org/user/adityaprasad1308/articles'  # creating request objectreq = requests.get(url)  # creating soup objectdata = BeautifulSoup(req.text, 'html')  # finding all li tags in ul and printing the text within itdata1 = data.find('ul')for li in data1.find_all("li"):    print(li.text, end=" ")
Output:
Picked
Python BeautifulSoup
Python bs4-Exercises
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Comments
Old Comments
Python Dictionary
Read a file line by line in Python
How to Install PIP on Windows ?
Enumerate() in Python
Iterate over a list in Python
Different ways to create Pandas Dataframe
Python String | replace()
Python program to convert a list to string
Create a Pandas DataFrame from Lists
Reading and Writing to text files in Python