Extract feed details from RSS in Python - GeeksforGeeks

24 Jan, 2021
In the article, we will be seeing how to extract feed and post details using RSS feed for a Hashnode blog. Although we are going to use it for blogs on Hashnode it can be used for other feeds as well.
RSS means Rich Site Summary and uses standard web formats to publish information that changes frequently like blog posts, news, audio, video, etc. RSS documents often know as feed which consists of text, and metadata, like time and author’s name.
Installing feed parser:
We will be using the Feedparser python library for parsing the RSS feed of the blog. It is quite a popular library for parsing blog feeds.
pip install feedparser
Let’s understand this stepwise:
Step 1: Getting RSS feed
Use the feedparser.parse() function for creating a feed object which contains parsed blog. It takes the URL of the blog feed.
Python3
# url of blog feedfeed_url = "https://vaibhavkumar.hashnode.dev/rss.xml"  blog_feed = feedparser.parse(feed_url)
Step 2: Getting details from the blog.
Python3
# returns title of the blog siteblog_feed.feed.title   # returns the link of the blog# and number of entries(blogs) in the site.blog_feed.feed.linklen(blog_feed.entries)  # Details of individual blog can# be accessed by using attribute nameprint(blog_feed.entries[0].title)print(blog_feed.entries[0].link)print(blog_feed.entries[0].author)print(blog_feed.entries[0].published)  # Getting lists of tags and authors.tags = [tag.term for tag in blog_feed.entries[0].tags]authors= [author.name for author in blog_feed.entries[0].authors]
Below is the full implementation: Now use the above code to write a function that takes the link of RSS feed and returns the details.
Python3
def get_posts_details(rss=None):        """    Take link of rss feed as argument    """    if rss is not None:                  # import the library only when url for feed is passed        import feedparser                  # parsing blog feed        blog_feed = blog_feed = feedparser.parse(rss)                  # getting lists of blog entries via .entries        posts = blog_feed.entries                  # dictionary for holding posts details        posts_details = {"Blog title" : blog_feed.feed.title,                        "Blog link" : blog_feed.feed.link}                  post_list = []                  # iterating over individual posts        for post in posts:            temp = dict()                          # if any post doesn't have information then throw error.            try:                temp["title"] = post.title                temp["link"] = post.link                temp["author"] = post.author                temp["time_published"] = post.published                temp["tags"] = [tag.term for tag in post.tags]                temp["authors"] = [author.name for author in post.authors]                temp["summary"] = post.summary            except:                pass                          post_list.append(temp)                  # storing lists of posts in the dictionary        posts_details["posts"] = post_list                   return posts_details # returning the details which is dictionary    else:        return None  if __name__ == "__main__":  import json    feed_url = "https://vaibhavkumar.hashnode.dev/rss.xml"    data = get_posts_details(rss = feed_url) # return blogs data as a dictionary      if data:    # printing as a json string with indentation level = 2    print(json.dumps(data, indent=2))   else:    print("None")
Output:
Web-scraping
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Python Dictionary
Read a file line by line in Python
How to Install PIP on Windows ?
Enumerate() in Python
Different ways to create Pandas Dataframe
Iterate over a list in Python
Python String | replace()
*args and **kwargs in Python
Reading and Writing to text files in Python
Create a Pandas DataFrame from Lists