Package: html2ans
This project provides a standardized method of parsing HTML elements intoANS elements. It is mainly used by Arc Publishing’s
professional services team to migrate client data into the Arc platform, but can also be
used for arbitrary conversion of HTML to JSON.html2ans is hosted onpypi.Please use theGitHub issue trackerto submit bugs or request features.Full documentation can be foundhere.QuickstartGenerating ANS from HTMLfromhtml2ans.defaultimportHtml2Ansparser=Html2Ans()content_elements=parser.generate_ans(your_html_here)Adding ParsersBasic AdditionIf you need to parse a certain tag in a customized way, you can write your own parser class and add it to the
parsersHtml2Answill use like so:fromhtml2ans.defaultimportHtml2Ansparser=Html2Ans()parser.add_parser(YourCustomImageParser())parser.generate_ans(your_html_here)The types of items your parser can parse should be listed in itsapplicable_elementsattribute.The default parser class (DefaultHtmlAnsParserorHtml2Ans) has parsers for text, links, images, various social media embeds, etc.Prioritized AdditionThe parsers that can be used for each element type (e.g.img,p) are held in a list. If you want your parser to have a higher priority than the default parsers, add it like so:fromhtml2ans.defaultimportHtml2Ansparser=Html2Ans()parser.insert_parser('img',YourCustomImageParser(),0)parser.generate_ans(your_html_here)Creating Custom ParsersMissing from the snippet above is a definition ofYourCustomImageParser. Before talking about how to create such a parser,
let’s examine why you might need to do so.The default image parserhtml2ans.parsers.image.ImageParserapplies to htmlimgtags only. Imagine you need to parse html whose images come indivtags (labelled with the classfancy-figure) that also hold a caption (labelled with the classfancy-caption). Here is a possible implementation of a parser for such images (note: this returns basic image ANS, not a reference):fromhtml2ans.parsers.imageimportImageParserfromhtml2ans.parsers.baseimportParseResultclassYourCustomImageParser(ImageParser):applicable_elements=['div','figure']applicable_classes=['fancy-figure']defparse(self,element,*args,**kwargs):image_tag=element.find('img')caption_tag=element.find('p',{"class":"fancy-caption"})ifimage_tag:image=self.construct_output(image_tag)ifcaption_tag:image["caption"]=caption_tag.textreturnParseResult(image,True)returnParseResult(None,True)Custom Parsing TipsANS VersionsSome ANS types require a version. You can set a version in your main parser (Html2Ans) and then automatically include that version in any element parser’s output by setting the parser’sversion_requiredattribute toTrue.Note: this doesn’t mean valid, version-compatible ANS is automatically produced!Keeping HTML intextOutputTo adjust what HTML is/isn’t left inline when parsing text, adjust theINLINE_TAGSattribute on the text parser. Every parser inherits fromhtml2ans.parsers.utils.AbstractParserUtilitieswhich provides a list of defaultINLINE_TAGSwhich can be used to make sure text formatters (e.g.strong,em, etc.) are left in place when text is parsed.Link ParsingBy default,atags are left inline in text, assuming there is text outside of the link. A link by itself (e.g.<p><ahref="google.com">Search</a></p>) will be turned into aninterstitial_link. Ifinterstitial_linkelements are unwanted, simply addato the list ofapplicable_elementsfor theParagraphParser.Removing Unnecessary TagsSometimes it is helpful to remove unnecessary tags (e.g.<p></p>,<div><imgsrc="..."/></div>). By default,Html2Ansconsiderspanddivtags with no attributes other thanid,class, orstyleto be unnecessary “wrappers”. When these are encountered, they are ignored and their children are parsed.The benefit of this is that<p></p>is ignored and<div><imgsrc="..."/></div>is parsed as an image.The downside is that sometimes you don’t want your HTML removed! There are a few options in this case. You can configure what tags can be considered wrappers via theWRAPPER_TAGSattribute onHtml2Ans. So ifdivtags should never be removed, simply removedivfrom this list. If a more complicated set of rules are necessary, override theis_wrappermethod onHtml2Ans.If it’s easier to modify the HTML than to modify this library, you can also add an arbitrary attribute like so:<divno_parse_flag="true">...</div>. Thisdivwill not be considered a wrapper when it is encountered.