Package: pantable
Date:November 30, 2021ContentsIntroductionA word on supportInstallationPipCondaNote on versionsSupported pandoc versionsPantable as pandoc filterspantableExampleUsageSyntaxpantable2csvpantable2csvxPantable as a libraryDevelopmentRelated Filtersdocstestspackageconda-forgeciteIntroductionPantable is a Python library that maps the pandoc Table AST to an internal structure losslessly. This enables writing pandoc filters specifically manipulating tables in pandoc.This also comes with 3 pandoc filters,pantable,pantable2csv,pantable2csvx.pantableis the main filter, introducing a syntax to include CSV table in markdown source. It supports all table features supported by the pandoc Table AST.pantable2csvcomplementspantable, is the inverse ofpantable, which convert native pandoc tables into the CSV table format defined bypantable. This is lossy as of pandoc 2.11+, which is supported since pantable 0.13.pantable2csvx(experimental, may drop in the future) is similar topantable2csv, but introduces an extra column with thefancy-tablesyntax defined below such that any general pandoc Table AST can be losslessly encoded in CSV format.Some example uses are:You already have tables in CSV format.You feel that directly editing markdown table is troublesome. You want a spreadsheet interface to edit, but want to convert it to native pandoc table for higher readability. And this process might go back and forth.You want lower-level control on the table and column widths.You want to use all table features supported by the pandoc’s internal AST table format, which is not possible in markdown for pandoc (as of writing.)A word on supportNote that the above is exactly how I use pantable personally. So you can count on the round-trip losslessness.pantableandpantable2csvshould have robust support since it has been used for years. But since pandoc 2.11 the table AST has been majorly revised. Pantable 0.13 added support for this new AST by completely rewriting pantable, at the same time addresses some of the shortcoming of the original design. Part of the new design is to enable pantable as a library (seePantable as a librarybelow) so that its functionality can be extended, similar to how to write a pandoc filter to intercept the AST and modify it, you can intercept the internal structure of PanTable and modify it.However, since this library is completely rewritten as of v0.13,pantableandpantable2csvas pandoc filters should be stablethere may be regression, please open an issue to report thisround-trip losslessness may break, please open an issue to report thispantable2csvxas pandoc filter is experimental. API here might change in the future or may be dropped completed (e.g. replaces by something even more general)Pantable as a library also is experimental, meaning that the API might be changed in the future.InstallationPipTo manage pantable using pip, open the command line and runpip install pantableto installpip installhttps://github.com/ickc/pantable/archive/master.zipto install the in-development versionpip install-Upantableto upgradepip uninstall pantableto removeYou need a matching pandoc version for pantable to work flawlessly. SeeSupported pandoc versionsfor details. Or, use theCondamethod to install below to have the pandoc version automatically managed for you.CondaTo manage pantablewith a matching pandoc version, open the command line and runconda install-cconda-forgepantableto installconda update pantableto upgradeconda remove pantableto removeYou may also replacecondabymamba, which is basically a drop-in replacement of the conda package manager. Seemamba-org/mamba: The Fast Cross-Platform Package Managerfor details.Note on versionsSupported Python versionspantable v0.12 drop Python 2 support. You need topip install pantable<0.12if you need to run it on Python 2.To enforce using Python 3, depending on your system, you may need to specifypython3andpip3explicitly.Check the badge above orsetup.pyfor supported Python versions,setup.pyfurther indicates support of pypy in additional of CPython.Supported pandoc versionspandoc versioning semantics isMAJOR.MAJOR.MINOR.PATCHand panflute’s is MAJOR.MINOR.PATCH. Below we shows matching versions of pandoc that panflute supports, in descending order. Only major version is shown as long as the minor versions doesn’t matter.Version Matching[1]pantablepanflute versionsupported pandoc versionssupported pandoc API versions0.14.12.1.32.11.0.4–2.16.x1.22–1.22.10.142.12.11.0.4—2.14.x1.220.132.02.11.0.4—2.11.x1.22not supported2.101.210.121.122.7-2.91.17.5–1.20Note: pandoc 2.10 is short lived and 2.11 has minor API changes comparing to that, mainly for fixing its shortcomings. Please avoid using pandoc 2.10.To use pantable with pandoc < 2.10, install pantable 0.12 explicitly bypip installpantable~=0.12.4.Pantable as pandoc filterspantableThis allows CSV tables, optionally containing markdown syntax (disabled by default), to be put in markdown as a fenced code blocks.ExampleAlso see the README inGitHub Pages.```table
---
caption: '*Awesome* **Markdown** Table'
alignment: RC
table-width: 2/3
markdown: True
---
First row,defaulted to be header row,can be disabled
1,cell can contain **markdown**,"It can be aribrary block element:

- following standard markdown syntax
- like this"
2,"Any markdown syntax, e.g.",E = mc^2^
```becomesAwesomeMarkdownTableFirst rowdefaulted to be header rowcan be disabled1cell can containmarkdownIt can be aribrary block element:following standard markdown syntaxlike this2Any markdown syntax, e.g.E = mc2(The equation might not work if you view this on PyPI.)Usagepandoc-Fpantable-oREADME.htmlREADME.mdSyntaxFenced code blocks is used, with a classtable. SeeExample.Optionally, YAML metadata block can be used within the fenced code block, following standard pandoc YAML metadata block syntax. 7 metadata keys are recognized:captionthe caption of the table. Can be block-like. If omitted, no caption will be inserted. Interpreted as markdown only ifmarkdown: truebelow.Default: disabled.short-captionthe short-caption of the table. Must be inline-like element. Interpreted as markdown only ifmarkdown: truebelow.Default: disabled.alignmentalignment for columns: a string of characters amongL,R,C,D, case-insensitive, corresponds to Left-aligned, Right-aligned, Center-aligned, Default-aligned respectively. e.g.LCRDfor a table with 4 columns.You can specify only the beginning that’s non-default. e.g.DLCRfor a table with 8 columns is equivalent toDLCRDDDD.Default:DDD...alignment-cellsalignment per cell. One row per line. A string of characters amongL,R,C,D, case-insensitive, corresponds to Left-aligned, Right-aligned, Center-aligned, Default-aligned respectively. e.g.LCRD
DRCLfor a table with 4 columns, 2 rows.you can specify only the top left block that is not default, and the rest of the cells with be default to default automatically. e.g.DC
LRfor a table with 4 columns, 3 rows will be equivalent toDCDD
LRDD
DDDDDefault:DDD...\n...widtha list of relative width corresponding to the width of each columns.Dmeans default width. e.g.-width- 0.1- 0.2- 0.3- 0.4- DAgain, you can specify only the left ones that are non-default and it will be padded with defaults.Default:[D, D, D,...]table-widththe relative width of the table (e.g. relative to\linewidth). If specified as a number, and if any of the column width inwidthis default, then auto-width will be performed such that the sum ofwidthequals this number.Default: NoneheaderIf it has a header row or not.Default: TruemarkdownIf CSV table cell contains markdown syntax or not.Default: Falsefancy_tableif true, then the first column of the table will be interpreted as a special fancy-table syntax s.t. it encodes which rows aretable-header,table-foot,multiple table-bodies and“body-head” within table-bodies.see example below.includethe path to an CSV file, can be relative/absolute. If non-empty, override the CSV in the CodeBlock.Default: Noneinclude-encodingif specified, the file fromincludewill be decoded according to this encoding, else assumed to be UTF-8. Hint: if you save the CSV file via Microsoft Excel, you may need to set this toutf-8-sig.csv-kwargsIf specified, should be a dictionary passed tocsv.readeras options. e.g.---csv-kwargs:dialect:unixkey:value......formatThe file format from the data in code-block or include if specified.Default:csvfor data from code-block, and infer from extension in include.Currently onlycsvis supported.ms(experimental, may drop in the future): a list of int that specifies the number of rows per row-block. e.g.[2, 6, 3, 4, 5, 1]means the table should have 21 rows, first 2 rows are table-head, last 1 row is table-foot, there are 2 table-bodies (indicated by6, 3, 4, 5in the middle) where the 1st body6, 3has 6 body-head and 3 “body-body”, and the 2nd body4, 5has 4 body-head and 5 “body-body”.If this is specified,headerwill be ignored.Default: None, which would be inferred fromheader.ns_head(experimental, may drop in the future): a list of int that specifies the number of head columns per table-body. e.g.[1, 2]means the 1st table-body has 1 column of head, the 2nd table-body has 2 column of headDefault: Nonepantable2csvThis one is the inverse ofpantable, a panflute filter to convert any native pandoc tables into the CSV table format used by pantable.Effectively,pantableforms a “CSV Reader”, andpantable2csvforms a “CSV Writer”. It allows you to convert back and forth between these 2 formats.For example, in the markdown source:+--------+---------------------+--------------------------+
| First  | defaulted to be     | can be disabled          |
| row    | header row          |                          |
+========+=====================+==========================+
| 1      | cell can contain    | It can be aribrary block |
|        | **markdown**        | element:                 |
|        |                     |                          |
|        |                     | -   following standard   |
|        |                     |     markdown syntax      |
|        |                     | -   like this            |
+--------+---------------------+--------------------------+
| 2      | Any markdown        | $$E = mc^2$$             |
|        | syntax, e.g.        |                          |
+--------+---------------------+--------------------------+

: *Awesome* **Markdown** Tablerunningpandoc-Fpantable2csv-ooutput.md input.md, it becomes``` {.table}
---
alignment: DDD
caption: '*Awesome* **Markdown** Table'
header: true
markdown: true
table-width: 0.8055555555555556
width: [0.125, 0.3055555555555556, 0.375]
---
First row,defaulted to be header row,can be disabled
1,cell can contain **markdown**,"It can be aribrary block element:

-   following standard markdown syntax
-   like this
"
2,"Any markdown syntax, e.g.",$$E = mc^2$$
```pantable2csvx(experimental, may drop in the future)Similar topantable2csv, but convert withfancy_tablesyntax s.t. any general Table in pandoc AST is in principle losslessly converted to a markdown-ish syntax in a CSV representation.e.g.pandoc-Fpantable2csvx-otests/files/native_reference/planets.mdtests/files/native/planets.nativewould turn the native Table fromplatnets.native[2]to``` {.table}
---
caption: Data about the planets of our solar system.
alignment: CCDRRRRRRRR
ns-head:
- 3
markdown: true
fancy-table: true
...
===,"(1, 2)
",,Name,Mass (10\^24kg),Diameter (km),Density (kg/m\^3),Gravity (m/s\^2),Length of day (hours),Distance from Sun (10\^6km),Mean temperature (C),Number of moons,Notes
,"(4, 2)
Terrestrial planets",,Mercury,0.330,"4,879",5427,3.7,4222.6,57.9,167,0,Closest to the Sun
,,,Venus,4.87,"12,104",5243,8.9,2802.0,108.2,464,0,
,,,Earth,5.97,"12,756",5514,9.8,24.0,149.6,15,1,Our world
,,,Mars,0.642,"6,792",3933,3.7,24.7,227.9,-65,2,The red planet
,"(4, 1)
Jovian planets","(2, 1)
Gas giants",Jupiter,1898,"142,984",1326,23.1,9.9,778.6,-110,67,The largest planet
,,,Saturn,568,"120,536",687,9.0,10.7,1433.5,-140,62,
,,"(2, 1)
Ice giants",Uranus,86.8,"51,118",1271,8.7,17.2,2872.5,-195,27,
,,,Neptune,102,"49,528",1638,11.0,16.1,4495.1,-200,14,
___,"(1, 2)
Dwarf planets",,Pluto,0.0146,"2,370",2095,0.7,153.3,5906.4,-225,5,Declassified as a planet in 2006.
```Pantable as a library(experimental, API may change in the future)Documentation here is sparse, partly because the upstream (pandoc) may change the table AST again. SeeCrazy ideas: table structure from upstream GitHub.See the API docs inhttps://ickc.github.io/pantable/.For example, looking at the source ofpantableas a pandoc filter, incodeblock_to_table.py, you will see the main function doing the work is nowpan_table_str=(PanCodeBlock.from_yaml_filter(options=options,data=data,element=element,doc=doc).to_pantablestr())ifpan_table_str.table_widthisnotNone:pan_table_str.auto_width()return(pan_table_str.to_pantable().to_panflute_ast())You can see another example fromtable_to_codeblock.pywhich is whatpantable2csvandpantable2csvxcalled.Below is a diagram illustrating the API:OverviewSolid arrows are lossless conversions. Dashed arrows are lossy.You can see the pantable internal structure,PanTableis one-one correspondence to the pandoc Table AST. Similarly forPanCodeBlock.It can then losslessly converts between PanTable and PanTableMarkdown, where everything in PanTableMarkdown is now markdown strings (whereas those in PanTable are panflute or panflute-like AST objects.)Lastly, it defines a one-one correspondence to PanCodeBlock withfancy_tablesyntax mentioned earlier.Below is the same diagram with the method names. You’d probably want to zoom into it to see it clearly.Detailed w/ methodsDevelopmentTo run all the tests runtox. GitHub Actions is used for CI too so if you fork this you can check if your commits passes there.Related Filters(The table here is created in the beginning of pantable, which has since added more features. This is left here for historical reason and also as a credit to those before this.)The followings are pandoc filters written in Haskell that provide similar functionality. This filter is born after testing with theirs.baig/pandoc-csv2table: A Pandoc filter that renders CSV as Pandoc Markdown Tables.mb21/pandoc-placetable: Pandoc filter to include CSV data (from file or URL)sergiocorreia/panflute/csv-tables.pypandoc-csv2tablepandoc-placetablepanflute examplepantablecaptioncaptioncaptiontitlecaptionalignsaligns = LRCDaligns = LRCDaligns = LRCDwidthwidths = “0.5 0.2 0.3”width: [0.5, 0.2, 0.3]table-widthtable-width: 1.0headerheader = yes | noheader = yes | nohas_header: True | Falseheader: True | False | yes | NOmarkdowninlinemarkdownmarkdown: True | False | yes | NOsourcesourcefilesourceincludeotherstype = simple | multiline | grid | pipedelimiterquotecharid (wrapped by div)Noteswidth are auto-calculated when width is not specified[1]For pandoc API verion, checkhttps://hackage.haskell.org/package/pandocfor pandoc-types, which is the same thing.[2]copied from pandoc fromhere, which was dual licensed as CC0here