Package: lectern
LecternLiterate Minecraft data packs and resource packs.@function tutorial:greetingsayHello,world!IntroductionThis markdown file is interspersed with code fragments describing the content of a Minecraft data pack. Usinglectern, you can turn this single file into an actual data pack that can be loaded into the game.FeaturesTurn markdown files into data packs and resource packsMerge resources from several markdown filesConvert data packs and resource packs into markdown snapshotsCan be used as abeetpluginHighly extensible with custom directivesAutomatically integrates withpytest-instaHmmkay but why?Editing data packs involves a lot of jumping around between files, for simple use-cases a single file is a lot easier to work withMinecraft packs aggregate various types of files that can have complex interactions with each other, a literate style allows you to document these interactions fluentlyHuman-readable, single-file data pack and resource pack snapshots can be really useful to diff and track regressions in Minecraft-related toolingInstallationThe package can be installed withpip.$pipinstalllecternGetting startedThis is an example of a markdown file that can be turned into a data pack:# Beginner tutorial

Let's start by creating a simple function:

`@function tutorial:greeting`

```mcfunction
say Hello, world!
```

And now we can make it run when the data pack is loaded!

`@function_tag minecraft:load`

```json
{
  "values": ["tutorial:greeting"]
}
```You can use thelecterncommand-line utility to turn the markdown file into a data pack.$lecterntutorial.md--data-packpath/to/tutorial_data_packIf you're usingbeetyou can uselecternas a plugin in your pipeline.{"pipeline":["lectern"],"meta":{"lectern":{"load":["*.md"]}}}Document formatslecternimplements two closely-related document formats: markdown and plain text. The markdown format builds upon the plain text format.The markdown format lets you present the various elements of your data pack or resource pack and how they fit together. It's a format that's meant to supportliterate programming. You can use it when your document is meant to be read by other people. It allows you to emphasize the important parts, explain tradeoffs and discuss alternatives, implementation details, etc...`@function tutorial:greeting`

```mcfunction
say Hello, world!
```On the other hand if you don't intend to produce literate documents you can use the plain text format to author data packs and resource packs as a single file without having to deal with markdown formatting.@function tutorial:greeting
say Hello, world!DirectivesData pack and resource pack fragments are code blocks, links or images annotated with a speciallecterndirective. Directives are prefixed with the@symbol and can be followed by zero or more arguments.@<directive_name> <arg1> <arg2> <arg3>...lecternprovides directives for including namespaced resources inside data packs and resource packs. These built-in directives all expect a single argument specifying the fully-qualified resource name.@function tutorial:greeting
@function_tag minecraft:loadHere is a reference of all the supported resources:Data packResource pack@advancement@blockstate@function@model@loot_table@language@predicate@font@recipe@glyph_sizes@structure@true_type_font@block_tag@shader_post@entity_type_tag@shader@fluid_tag@fragment_shader@function_tag@vertex_shader@game_event_tag@glsl_shader@item_tag@text@dimension_type@texture_mcmeta@dimension@texture@biome@sound@configured_carver@particle@configured_feature@configured_structure_feature@configured_surface_builder@noise_settings@processor_list@template_pool@item_modifierNote that these directives are resolved automatically. If you're working with pack extensions your custom namespaced resources will have their own directives as well.There are also two built-in directives that can be used to include files using a path relative to the root of the data pack or the resource pack.@data_pack pack.mcmeta
@resource_pack pack.png
@resource_pack assets/minecraft/textures/block/kelp_plant.png.mcmetaThis is useful for adding files that aren't part of any particular namespace.In case you need to bundle existing resource packs or data packs, you can use the@merge_zipdirective.@merge_zip(download)
https://example.com/my_zipped_data_pack.zipFinally, the@skipdirective is simply ignored and allows you to end a previous fragment in the plain text format.@function tutorial:greeting
say Hello, world!

@skip
This will not be included in the output.Code block fragmentsYou can include the content of a code block in a data pack or a resource pack by preceding it with a directive surrounded by backticks.@function tutorial:greetingsayHello,world!You can put the directive in an html comment to make it invisible. Here the code block is annotated with the following comment:<!-- @function_tag minecraft:load -->{"values":["tutorial:greeting"]}When using backticks you can surround the code block in a<details>element to make the code fragment foldable.@function tutorial:greetingsayHello,world!The directive can also be embedded directly inside the code block. You can insert a directive preceded by either#or//and the following lines will be included in the specified file.# @function tutorial:obtained_dead_bushsayYouobtainedadeadbush!Embedded directives are stripped from the output. You can use multiple directives in a single code block.// @loot_table minecraft:blocks/diamond_ore{"pools":[{"rolls":1,"entries":[{"type":"minecraft:item","name":"minecraft:dead_bush"}]}]}// @advancement tutorial:obtained_dead_bush{"criteria":{"dead_bush":{"trigger":"minecraft:inventory_changed","conditions":{"items":[{"item":"minecraft:dead_bush"}]}}},"requirements":[["dead_bush"]],"rewards":{"function":"tutorial:obtained_dead_bush"}}It's also possible to use the lectern text format directly inside code blocks.@functiontext_in_block:foosayfooLink fragmentsLink fragments make it possible to refer to external files, online assets, and to embed binary files in the markdown as data urls. You can create a link fragment by turning a directive surrounded by backticks into a markdown link.@loot_table minecraft:blocks/yellow_shulker_boxThe link itself can be a path to a local file or any url supported by the built-inurlopenfunction.Image fragmentsYou can include inline markdown images in the output data pack or resource pack by preceding the image with a directive surrounded by backticks.@data_pack pack.pngImage fragments support the same variations as code block fragments. You can put the directive in a comment or surround the image with a<details>element to make it foldable.Hidden fragmentsYou can use html comments to add fragments that will be completely hidden in the rendered markdown.<!--@function tutorial:hiddensay This will not appear in the rendered markdown.@function tutorial:also_hiddensay This is also hidden.-->ModifiersThe behavior of particular directives can be adjusted with modifiers. A modifier is specified between parentheses right after the name of the directive.@<directive_name>(<modifier>) <arg1> <arg2> <arg3>...Theappendmodifier is implemented by all the text-based built-in namespaced resource directives and makes it possible to concatenate the content of the fragment to the already-existing content.@function(append) tutorial:greetingsayThisisaddedafterwards.You can also useprependto add the fragment before the already-existing content.@function(prepend) tutorial:greetingsayThisisaddedbefore.Themergemodifier is similar but instead of concatenating the contents it uses thebeetmerging strategy to combine the fragment with the existing file.@function_tag(merge) minecraft:load{"values":["#tutorial:something_else"]}There are also modifiers that are applied to the content of the fragment directly. Thebase64modifier will decode the content of the code fragment asbase64.@function_tag(base64) tutorial:something_elseewogICJ2YWx1ZXMiOiBbInR1dG9yaWFsOnN0cmlwcGVkIl0KfQ==You can use block fragments to download remote files with thedownloadmodifier.@function_tag(download) tutorial:from_githubhttps://raw.githubusercontent.com/mcbeet/beet/main/examples/load_basic/src/data/demo/functions/foo.mcfunctionFinally, there's astrip_final_newlinemodifier that removes the final newline at the end of code block fragments. It's mostly used to make sure thatlecternsnapshots can reconstruct the original content byte for byte in case the file wasn't terminated by a newline.@function(strip_final_newline) tutorial:strippedsayThisfunctiondoesn't have a final newline.OverlaysYou can use the@overlaydirective to make the following directives apply to a specific pack overlay. Overlays were introduced inJava Edition 1.20.2.@overlay({"min_inclusive": 16, "max_inclusive": 17}) dummy_overlayYou can specify theformatssupported by this overlay as a modifier. From now on, all the directives will apply to the overlaydummy_overlay.@function tutorial:greetingsayHellofromoverlay!You can switch to another overlay at any time by using the@overlaydirective again. To go back to the main pack, use the@endoverlaydirective.@endoverlayCommand-line utility$lectern--help
Usage:lectern[OPTIONS][PATH]...LiterateMinecraftdatapacksandresourcepacks.

Options:-d,--data-pack<path>Extractdatapack.-r,--resource-pack<path>Extractresourcepack.-e,--external-files<path>Emitexternalfiles.-p,--prefetch-urls<path>Prefetchmarkdownlinks.-f,--flatUsetheflatmarkdownformat.-o,--overwriteOverwritetheoutputpack.-v,--versionShowtheversionandexit.-h,--helpShowthismessageandexit.You can extract data packs from markdown files with the-d/--data-packoption. If the name ends with.zipthe generated data pack will be zipped. Multiple markdown files can be merged together into a single data pack.$lecterndemo.md--data-packdemo_data_pack
$lecterndemo.md-ddemo_data_pack
$lecterndemo.md-ddemo_data_pack.zip
$lecternfoo.mdbar.md-ddemo_data_packThe-r/--resource-packoption lets you do exactly the same thing but with resource packs. The two options can be combined to extract a data packs and a resource pack at the same time.$lecterndemo.md--resource-packdemo_resource_pack
$lecterndemo.md-rdemo_resource_pack
$lecterndemo.md-ddemo_data_pack-rdemo_resource_packIf you want to overwrite an existing data pack or resource pack you need to specify the-o/--overwriteoption explicitly.$lecterndemo.md--overwrite--data-packdemo_data_packYou can also convert a combination of data packs and resource packs into a single markdown file.$lecterndemo_data_packdemo.md
$lecterndemo_data_pack.zipdemo.md
$lecterndemo_data_packdemo_resource_packdemo.md
$lecternfoo_data_packbar_data_packdemo.mdThe last argument is the name of the generated markdown file. By default, thelecternutility will bundle binary files into the markdown file as data urls. You can use the-e/--external-filesoption to dump the binary files in a given directory instead.$lecterndemo_data_packdemo.md--external-filesfiles
$lecterndemo_data_packdemo.md-efiles
$lecterndemo_data_packdemo.md-e.All these commands also work with plain text files.lecternwill only use the markdown document format when the filename ends with.md.Finally, you can also use the command-line utility to prefetch markdown urls. The-p/--prefetch-urlsoption can replace the urls in-place or in a copy.$lectern--prefetch-urlsdemo.md
$lectern--prefetch-urlsdemo.mddemo_prefetched.md
$lectern-pdemo.mddemo_prefetched.md
$lectern-pdemo.mdBy default, the remote files will be bundled as data urls but you can use the-e/--external-filesoption to dump everything in a given directory.$lectern--prefetch-urlsdemo.md--external-filesfiles
$lectern--prefetch-urlsdemo.mddemo_prefetched.md--external-filesfiles
$lectern-pdemo.mddemo_prefetched.md-efiles
$lectern-pdemo.md-e.Python APIThe API revolves aroundDocumentobjects. Alecterndocument holds aDataPackand aResourcePack, as well as a dictionary defining the usable directives. The extractors and serializers are also exposed on the document to make it possible to swap them out with custom ones if needed.frombeetimportDataPack,ResourcePackfromlecternimportDocumentdocument=Document()assertdocument.data==DataPack()assertdocument.assets==ResourcePack()The constructor makes it possible to provide existingDataPackandResourcePackinstances, some initial text or markdown content, or a path from which to load an existinglecterndocument.Document(data=DataPack(),assets=ResourcePack())Document(text=...)Document(markdown=...)Document(path="path/to/document.md")Documentinstances will compare equal if the underlying data packs and resource packs also compare equal.You can use theloadmethod to read a markdown or a plain text file and update the internal data pack and resource pack with the extracted fragments.document.load("path/to/document.md")If you already have some text or markdown ready to go, you can use theadd_textandadd_markdownmethods.document.add_text(...)document.add_markdown(...)If the markdown content refers to local files you can specify the directory from which the external files should be loaded from with theexternal_filesargument.document.add_markdown(...,external_files="path/to/directory")If you're handling user input and you don't know which document format is being provided you can use theaddmethod andlecternwill detect if the input is markdown or plain text.document.add(...)You can use theget_textandget_markdownmethods to serialize the entire content of the internal data pack and resource pack. By default theget_markdownmethod will produce markdown that embeds binary files as data urls. You can enableemit_external_filesand optionally provide a path prefix to generate a dictionary of associated files instead.text=document.get_text()markdown=document.get_markdown()markdown,external_files=document.get_markdown(emit_external_files=True)markdown,external_files=document.get_markdown(emit_external_files=True,prefix="path/to/directory")Finally, thesavemethod lets you serialize and write the document to a given path. If the filename ends with.mdthe generated markdown will bundle binary files as data urls by default. You can use theexternal_filesargument to emit the binary files in the given directory instead.document.save("path/to/document.txt")document.save("path/to/document.md")document.save("path/to/document.md",external_files="path/to/files")Custom directivesDirectives are simply callable objects that receive the document fragment, the resource pack, and the data pack as arguments.frombeetimportDataPack,ResourcePack,FunctionfromlecternimportDocument,Fragmentdefmy_directive(fragment:Fragment,assets:ResourcePack,data:DataPack):num1,num2=fragment.expect("num1","num2")result=int(num1)+int(num2)data["demo:output_result"]=Function([f"say{result}"])document=Document()document.directives["my_directive"]=my_directivedocument.add_text("@my_directive 32 10")assertdocument.data.functions["demo:output_result"]==Function(["say 42"])Theexpectmethod allows you to unpack the directive arguments and automatically raises an error if the user didn't specify the arguments properly. You can use theas_filemethod to get the content of the fragment as a specific type of file.defrepeated_function(fragment:Fragment,assets:ResourcePack,data:DataPack):full_name,count=fragment.expect("full_name","count")function=fragment.as_file(Function)function.lines*=int(count)data[full_name]=functionTheas_filemethod will take care of reading the file or downloading it if the directive is used with a link fragment. It will also handle thebase64andstrip_final_newlinemodifiers.You can handle custom modifiers by checking the content of themodifierattribute.Fragment loadersTheDocumentobject lets you register fragment loaders that intercept and potentially modify fragments before they're handled by directives. Theloadersattribute is a list of callable objects that receive the fragment and the available directives as arguments. Each loader can then forward the fragment as-is or return a modified fragment. You can also returnNoneto drop the fragment.fromtypingimportMapping,OptionalfromlecternimportDirective,Document,Fragmentdefhandle_ignore_modifier(fragment:Fragment,directives:Mapping[str,Directive],)->Optional[Fragment]:iffragment.modifier=="ignore":returnNonereturnfragmentdocument=Document()document.loaders.append(handle_ignore_modifier)document.add_text("@function(ignore) demo:foo\nsay hello")assertnotdocument.dataBeet pluginUsinglecternas abeetplugin makes it possible to combine your markdown files with arbitrarybeetplugins for further processing. The plugin can load files using the plain text and markdown document formats and emit a snapshot of thebeetcontext at the end of the build.{"pipeline":["lectern"],"meta":{"lectern":{"load":["*.md"],"snapshot":"out/snapshot.md","external_files":"out"}}}You can require the plugin programmatically by using thelecternplugin factory.frombeetimportContextfromlecternimportlecterndefmy_plugin(ctx:Context):ctx.require(lectern(load=["*.md"],snapshot="out/snapshot.md",external_files="out",))All the configuration is optional. The plugin is a no-op if theloadorsnapshotoptions are not specified.You can retrieve theDocumentinstance with theinjectmethod. This is useful for adding custom directives.frombeetimportContext,DataPack,ResourcePack,FunctionfromlecternimportDocument,Fragmentdefhello_directive(ctx:Context):"""Plugin that defines the `@hello <name>` directive."""document=ctx.inject(Document)document.directives["hello"]=hellodefhello(fragment:Fragment,assets:ResourcePack,data:DataPack):name=fragment.expect("name")function=data.functions.setdefault("hello:greetings",Function([]))function.lines.append(f"say Hello,{name}!")It's worth mentioning thatlecternuses thebeetcache to avoid downloading link fragments repeatedly and keeping your build snappy, especially in watch mode. If you need to re-download link fragments you can clear thelecterncache.$beetcache--clearlecternYou can also use a plugin to configure a custom cache timeout if you want to make sure that your assets are re-downloaded periodically.frombeetimportContextdefdownload_every_day(ctx:Context):ctx.cache["lectern"].timeout(hours=24)Extra directivesIf you're usinglecternas abeetplugin you will be able to use additional directives by adding the corresponding plugins to therequireoption.{"require":["lectern.contrib.require","lectern.contrib.script","lectern.contrib.define"],"pipeline":["lectern"],"meta":{"lectern":{"load":["*.md"]}}}Thelectern.contrib.requireplugin adds a directive that lets you require plugins dynamically.@require my_plugins.helloThelectern.contrib.scriptplugin adds a directive that renders a fragment with Jinja and interprets the result aslecterntext.@script{% for i in range(10) %}
@function demo:script_{{ i }}
say {{ i }}
{% endfor %}Note that using@scriptwith the text format requires you to escape the directives in the fragment with an additional@symbol.@script
{% for i in range(10) %}
@@function demo:script_{{ i }}
say {{ i }}
{% endfor %}Thelectern.contrib.defineplugin adds a directive that renders a fragment with Jinja and stores the resulting string as a template global.@define(strip_final_newline) math_message2 + 2 is {{ 2 + 2 }}Lectern scriptsThe text format is pretty well-suited for writing basic data pack and resource pack generators. You can very easily write scripts that produce lectern syntax that can then be turned into a data pack or a resource pack.# my_script.pyprint("@function tutorial:count")foriinrange(10):print(f"say{i}")$pythonmy_script.py>output.txt
$lecternoutput.txt-dmy_data_packThebeetplugin supports this use-case natively and allows you to run lectern scripts within your pipeline.// beet.json{"pipeline":["lectern"],"meta":{"lectern":{"scripts":[["python","my_script.py"]]}}}Thescriptsoption lets you specify the command-line arguments for your scripts. The scripts don't even have to be written in Python. As long as the command prints out validlecternsyntax you can use any language you want.Relative resource locationsThelectern.contrib.relative_locationplugin uses thebeetcontext generator to generate namespaced resources in a default location automatically if you don't specify any namespace.@function foo
function tutorial:bar

@function bar
say helloYou can customize the root of unqualified resource locations by using thegenerate_namespaceandgenerate_prefixmeta variables. By default,generate_namespaceis set to theproject_idandgenerate_prefixis empty.The plugin works pretty nicely withbeet.contrib.relative_function_path.// beet.json{"require":["lectern.contrib.relative_location"],"pipeline":["lectern","beet.contrib.relative_function_path"],"meta":{"lectern":{"load":["*.md"]}}}@function foo
function ./bar

@function bar
say helloUsing YAMLYou can use thelectern.contrib.yaml_to_jsonplugin to author JSON files with YAML. Since YAML is a superset of JSON, you don't have to do anything when you use the plugin. YAML fragments are transparently converted to JSON.{"require":["lectern.contrib.yaml_to_json"],"pipeline":["lectern"],"meta":{"lectern":{"load":["*.md"]}}}@function_tag minecraft:tick
values:
  - demo:fooThe@data_packand@resource_packdirectives will also convert the fragment to JSON if the file extension matches.ymlor.yaml.@resource_pack assets/minecraft/sounds.yml
block.note_block.bit_1:
  sounds:
    - block/note_block/bit_1
  subtitle: subtitles.block.note_block.noteSnapshot testingA lot of Minecraft tooling involves generating data packs and resource packs. Writing tests for this kind of tooling takes time because you need to painstakingly compare everything that you care about with a reference value. This makes it hard to get good coverage, and then even harder to keep making changes to the code being tested afterwards. You're trading robustness and stability for a shackle that massively slows down development.That's where snapshot testing comes into play. Snapshot testing allows you to record a reference value and then make sure that your code keeps producing the same results. It provides the necessary tools for reviewing snapshots and updating them as your project evolves.lecterndocuments are useful as snapshot formats because they represent entire data packs and resource packs in a single file that's human-readable and diff-friendly.pytest-instais an extensible snapshot testing plugin forpytest. When it's installed,lecterndefines three additional snapshot formats.ExtensionFormat description.pack.txtPlain text snapshot..pack.mdMarkdown snapshot with bundled binary files..pack.md_external_filesDirectory with a README.md that refers to external binary files.You can use these snapshot formats when comparingDocumentobjects with thesnapshotfixture.deftest_generate(snapshot):data=generate_some_data_pack()assertsnapshot("pack.txt")==Document(data=data)If you're using thebeettoolchain, keep in mind that you can get aDocumentinstance bound to the context object by using theinjectmethod.deftest_generate_with_beet(snapshot):withrun_beet(...)asctx:assertsnapshot("pack.txt")==ctx.inject(Document)This will save the entire data pack and resource pack in the snapshot. For more details about working with the generated snapshots check out thepytest-instadocumentation.ContributingContributions are welcome. Make sure to first open an issue discussing the problem or the new feature before creating a pull request. The project usespoetry.$poetryinstallYou can run the tests withpoetry run pytest.$poetryrunpytestThe project must type-check withpyright. If you're using VSCode thepylanceextension should report diagnostics automatically. You can also install the type-checker locally withnpm installand run it from the command-line.$npmrunwatch
$npmruncheckThe code follows theblackcode style. Import statements are sorted withisort.$poetryrunisortlecterntests
$poetryrunblacklecterntests
$poetryrunblack--checklecterntestsLicense -MIT