Package: meta-tags-parser
Meta tags parserFast, modern, pure python meta tags parser and snippet creator with full support of type annotations, py.typed in basic package and structured output. No jelly dicts, only typed structures!If you want to see what exactly is social media snippets, look at the example:RequirementsPython 3.8+HttpxInstallpip install meta-tags-parserUsageTL:DRParse meta tags from source:frommeta_tags_parserimportparse_meta_tags_from_source,structsdesired_result:structs.TagsGroup=parse_meta_tags_from_source("""... html source ...""")# desired_result — is what you wantParse meta tags from url:frommeta_tags_parserimportparse_tags_from_url,parse_tags_from_url_async,structsdesired_result:structs.TagsGroup=parse_tags_from_url("https://xfenix.ru")# and async variantdesired_result:structs.TagsGroup=awaitparse_tags_from_url_async("https://xfenix.ru")# desired_result — is what you want for both casesParse social media snippet from source:frommeta_tags_parserimportparse_snippets_from_source,structssnippet_obj:structs.SnippetGroup=parse_snippets_from_source("""... html source ...""")# snippet_obj — is what you want# access like snippet_obj.open_graph.title, ...Parse social media snippet from url:frommeta_tags_parserimportparse_snippets_from_url,parse_snippets_from_url_async,structssnippet_obj:structs.SnippetGroup=parse_snippets_from_url("https://xfenix.ru")# and async variantsnippet_obj:structs.SnippetGroup=awaitparse_snippets_from_url_async("https://xfenix.ru")# snippet_obj — is what you want# access like snippet_obj.open_graph.title, ...Huge note: functions*_from_urlwritten only for convenience and very error-prone, so any reconnections/error handling — completely on your side.Also, I don't want to add some bloated requirements to achieve robust connections for any users, because they may simply not await any of this from the library. But if you really need this — write me.Basic snippets parsingLets say you want extract snippet for twitter from html page:frommeta_tags_parserimportparse_snippets_from_source,structsmy_result:structs.TagsGroup=parse_snippets_from_source("""<meta property="og:card" content="summary_large_image"><meta property="og:url" content="https://github.com/"><meta property="og:title" content="Hello, my friend"><meta property="og:description" content="Content here, yehehe"><meta property="twitter:card" content="summary_large_image"><meta property="twitter:url" content="https://github.com/"><meta property="twitter:title" content="Hello, my friend"><meta property="twitter:description" content="Content here, yehehe">""")print(my_result)# What will be printed:"""SnippetGroup(open_graph=SocialMediaSnippet(title='Hello, my friend',description='Content here, yehehe',image='',url='https://github.com/'),twitter=SocialMediaSnippet(title='Hello, my friend',description='Content here, yehehe',image='',url='https://github.com/'))"""# You can access attributes as thismy_result.open_graph.titlemy_result.twitter.image# All fields are necessary and will be always available, even if they have not contain data# So no need to worry about attributes exsitence (but you may need to check values)Basic meta tags parsingMain function isparse_meta_tags_from_source. It can be used like this:frommeta_tags_parserimportparse_meta_tags_from_source,structsmy_result:structs.TagsGroup=parse_meta_tags_from_source("""... html source ...""")print(my_result)# What will be printed:"""structs.TagsGroup(title="...",twitter=[structs.OneMetaTag(name="title", value="Hello",...)],open_graph=[structs.OneMetaTag(name="title", value="Hello",...)],basic=[structs.OneMetaTag(name="title", value="Hello",...)],other=[structs.OneMetaTag(name="article:name", value="Hello",...)])"""As you can see from this example, we are not using any jelly dicts, only structured dataclasses. Lets see another example:frommeta_tags_parserimportparse_meta_tags_from_source,structsmy_result:structs.TagsGroup=parse_meta_tags_from_source("""<meta property="twitter:card" content="summary_large_image"><meta property="twitter:url" content="https://github.com/"><meta property="twitter:title" content="Hello, my friend"><meta property="twitter:description" content="Content here, yehehe">""")print(my_result)# What will be printed:"""TagsGroup(title='',basic=[],open_graph=[],twitter=[OneMetaTag(name='card', value='summary_large_image'),OneMetaTag(name='url', value='https://github.com/'),OneMetaTag(name='title', value='Hello, my friend'),OneMetaTag(name='description', value='Content here, yehehe')],other=[])"""forone_taginmy_result.twitter:ifone_tag.name=="title":print(one_tag.value)# What will be printed:"""Hello, my friend"""If you want to improve speedYou can specify what you want to parse:frommeta_tags_parserimportparse_meta_tags_from_source,structsresult:structs.TagsGroup=parse_meta_tags_from_source("""... source ...""",what_to_parse=(WhatToParse.TITLE,WhatToParse.BASIC,WhatToParse.OPEN_GRAPH,WhatToParse.TWITTER,WhatToParse.OTHER))If you reduce this tuple of parsing requirements it may increase overall parsing speed.Important notesAny name in meta tag (name or property attribute) will be lowercasedI decided to stripog:andtwitter:from original attributes, and let dataclass structures carry this information. If parser met meta tag with propertyog:name, it will be available inmy_resultvariable as one element of listmy_result.open_graphTitle of page (e.g.<title>Something</title>) will be available as stringmy_result.title(of course, you recieveSomething)«Standart» tags like title, description (check full list here./meta_tags_parser/structs.pyin constantBASIC_META_TAGS) will be available as list inmy_result.basicOther tags will be available as list inmy_result.otherattribute, name of tags will be preserved, unlikeog:/twitter:behaviourIf you want structured snippets, useparse_snippets_from_sourcefunctionChangelogYou can checkhttps://github.com/xfenix/meta-tags-parser/releases/release page.