Package: markpickle
markpickleMarkpickle is a Python library for lossy serialization of markdown to simple python data types and back. Imagine if markdown headers were used to define nested dictionaries and Markdown lists were python lists.It will create predictable markdown from a python object, but can't turn all markdown files into sensible python objects (for that use a markdown library that creates an AST). I created this because I wanted a way to turn json into Markdown. It is an accidental successor tomarkdown-to-json.-1-2becomes the python list[1, 2]# Cat## NameRingo## SpeciesFelixbecomes the python list{"cat":{"Name":"Ringo","Species":"Felix"}Seeexamplesfor representable types.Almost all markdown libraries use it as a way to generate HTML fragments from untrusted sources for insertion into some other HTML template. We are using it to represent data. Seeguidancefor which library make sense for you.InstallationpipinstallmarkpickleCapabilitiesThis is a lossy serialization. Markdown is missing too many concepts to make a high fidelity representation of a python data structure. If you want an object model that faithfully represents each object in a Markdown document, use the AST of mistune or one of the other markdown parsers.Supported TypesScalar valuesLists of scalar valuesDictionaries with scalar valuesLists of dictionaries of scalar valuesDictionaries with list valuesPartial support for blanks/string with leading/trailing whitespaceSeeexamples.Not SupportedThings not ordinarily serializableMarkdown that uses more than headers, lists, tablesBlanks, falsy values, empty iterables don't round tripScalar type inference doesn't round trip. After a scalar is converted to a markdown string, there is no indication if the original was a string or not.Serializing and DeserializingSerializingResults can be formatted at the cost of speed. Dictionaries can be represented as tables or header-text pairs.DeserializingMarkdown is deserialized by parsing the document to an abstract syntax tree (AST). This is done bymistune. If the markdown file has the same structure that markpickle uses, then it will create a sensible object. Deserializing a random README.md file is not expected to always work. For that, you should use mistune's AST.Round TrippingSome but not all data structures will be round-trippable. The goal is that the sort of dicts you get from loading JSON will be round-trippable, provided everything is a string.Splitting FilesIf typical serialization scenarios, many json files might be written to a single file, or in the case of yaml, you can put multiple documents into one file separated by---. markpickle can treat the horizontal rule as a document spliter if you usesplit_file. It works likesplitstream, but less efficiently.CLIThis command will take a deserializable markdown file and output json.python-mmarkpickle"docs/individual/list of scalars.md"Prior ArtImagine you have json and want to the same data as markdown. Json looks like python dict, so any python library that can convert json to markdown, probably can convert a python dict to markdown.Many tools turn tabular data into a markdown table.Serializing to Markdownjson2md, a node package, will turn json that looks like the HTML document object model into markdown, e.g.{"h1":"Some Header","p":"Some Text"}tomarkwill turn dict into a markdown table. Unmaintained.pytablewriteralso, dict to table, but supports many tabular formats.Deserializing to PythonMost libraries turn markdown into document object model. Markdown-to-json is the most similar to markpickle's goal of turning a markdown document into a python data types, in this case nested dicts.markdown-to-jsonis the library most similar to markpickle and is now maintained. It handles only deserialization and conversion to json.mistunewill turn markdown into an Abstract Syntax Tree. The AST is faithful representation of the Markdown, including concepts that have no semantic equivalent to python datatypes.beautifulsoupwill let you navigate the HTML DOM. So you can turn the markdown into HTML, then parse with Beautiful Soup.Representable TypesThere is one optional root dictionary representable with ATX headers, e.g.#,##, etc. Lists are nestable lists or dicts. For the most part, this looks like the types that JSON can represent.SerializableTypes:TypeAlias=Union[ColumnsValuesTableType,dict[str,"SerializableTypes"],list["SerializableTypes"],str,int,float,bool,datetime.date,None,]The deserialized types is the same except all Scalars are strings.Schema Validation for MarkdownIn the case of a serialization library, you'd want something that would indicate if your markdown file will successfully deserialize back into python.I haven't really found anything that says, for example, "This markdown document shall have one # Header and a 3 column table and nothing else."schema-markdown-jsA json schema that happens to be using markdown as its syntax.CreditsI copied the ATX-dictionary-like header parsing frommarkdown-to-json.DocumentationChoosing a LibraryExamplesTODOPeople solving similar problems on StackOverflowChange Log0.2.0 - Idea and reserve package name.1.0.0 - Basic functionality.1.1.0 - Basic functionality.1.2.0 - Add support for binary data, which is serialized as images with data URLs.1.3.0 - Improve CLI and more examples1.4.0 - ATX as dictionary now works1.5.0 - Tables less buggy1.5.1 - Fix mypy typing. Pin mistune to <3.0.0