Package: rss-parser
Rss parserAboutrss-parseris typed python RSS/Atom parsing module built usingpydanticandxmltodictInstallationpipinstallrss-parserorgitclonehttps://github.com/dhvcc/rss-parser.gitcdrss-parser
poetrybuild
pipinstalldist/*.whlV1 -> V2 migrationParserclass was renamed toRSSParserModels for RSS-specific schemas were moved fromrss_parser.modelstorss_parser.models.rss. Generic types are not touchedDate parsing was changed a bit, now uses pydantic'svalidatorinstead ofemail.utils, so the code will produce datetimes better, where it was defaulting tostrbeforeUsageQuickstartNOTE: For parsing Atom, useAtomParserfromrss_parserimportRSSParserfromrequestsimportget# noqarss_url="https://rss.art19.com/apology-line"response=get(rss_url)rss=RSSParser.parse(response.text)# Print out rss meta dataprint("Language",rss.channel.language)print("RSS",rss.version)# Iteratively print feed itemsforiteminrss.channel.items:print(item.title)print(item.description[:50])# Language en# RSS 2.0# Wondery Presents - Flipping The Bird: Elon vs Twitter# <p>When Elon Musk posted a video of himself arrivi# Introducing: The Apology Line# <p>If you could call a number and say you’re sorryHere we can see that description is still somehow has- this is beacause it's placed asCDATAlike so<![CDATA[<p>If you could call ...</p>]]>Overriding schemaIf you want to customize the schema or provide a custom one - useschemakeyword argument of the parserfromrss_parserimportRSSParserfromrss_parser.modelsimportXMLBaseModelfromrss_parser.models.rssimportRSSfromrss_parser.models.typesimportTagclassCustomSchema(RSS,XMLBaseModel):channel:None=None# Removing previous channel fieldcustom:Tag[str]withopen("tests/samples/custom.xml")asf:data=f.read()rss=RSSParser.parse(data,schema=CustomSchema)print("RSS",rss.version)print("Custom",rss.custom)# RSS 2.0# Custom Custom tag dataxmltodictThis library usesxmltodictto parse XML data. You can see the detailed documentationhereThe basic thing you should know is that your data is processed into dictionariesFor example, this data<tag>content</tag>will result in the following{"tag":"content"}But, when handling attributes, the content of the tag will be also a dictionary<tagattr="1"data-value="data">data</tag>Turns into{"tag":{"@attr":"1","@data-value":"data","#text":"content"}}Multiple children of a tag will be put into a list<div><tag>content</tag><tag>content2</tag></div>Results in a list[{"tag":"content"},{"tag":"content"},]If you don't want to deal with those conditions and parse somethingalwaysas a list -
please, userss_parser.models.types.only_list.OnlyListlike we did inChannelfromtypingimportOptionalfromrss_parser.models.rss.itemimportItemfromrss_parser.models.types.only_listimportOnlyListfromrss_parser.models.types.tagimportTagfromrss_parser.pydantic_proxyimportimport_v1_pydanticpydantic=import_v1_pydantic()...classOptionalChannelElementsMixin(...):...items:Optional[OnlyList[Tag[Item]]]=pydantic.Field(alias="item",default=[])Tag fieldThis is a generic field that handles tags as raw data or a dictonary returned with attributesExamplefromrss_parser.modelsimportXMLBaseModelfromrss_parser.models.types.tagimportTagclassModel(XMLBaseModel):width:Tag[int]category:Tag[str]m=Model(width=48,category={"@someAttribute":"https://example.com","#text":"valid string"},)# Content value is an integer, as per the generic typeassertm.width.content==48asserttype(m.width),type(m.width.content)==(Tag[int],int)# The attributes are empty by defaultassertm.width.attributes=={}# But are populated when provided.# Note that the @ symbol is trimmed from the beggining and name is convert to snake_caseassertm.category.attributes=={'some_attribute':'https://example.com'}ContributingPull requests are welcome. For major changes, please open an issue first
to discuss what you would like to change.Install dependencies withpoetry install(pip install poetry)pre-commitusage is highly recommended. To install hooks runpoetryrunpre-commitinstall-t=pre-commit-t=pre-pushLicenseGPLv3