Package: aocd-example-parser
Advent-of-Code-Data Example ParserThe annual programming challengeAdvent of Codefrequently provides some example data (and solutions) for you to test your code against, which are usually scaled-down versions of the real input data you're supposed to be solving with each day.Although the real puzzle inputs and answers differ by user, the example data is written directly in the puzzle prose and is available to unauthenticated users too.To illustrate what this means, the first puzzle of 2022 was--- Day 1: Calorie Counting ---and it has the following example data (54 bytes):1000
2000
3000

4000

5000
6000

7000
8000
9000

10000The correct answers corresponding to this sample data were24000for part a, and45000for part b. The example data text was found in the first<pre>tag, and the answers were found in the last<code>blocks from each of the two<article>sections in the puzzle page HTML. Note that the first half of the puzzle must be solved before the second<article>tag shows up, so the second article is not visible to unauthenticated users.In 2022, this same pattern of example data locations in the HTML was repeated for four more days, consecutively, before varying on--- Day 6: Tuning Trouble ---.What is this package?This package provides an implementation of anaocd example parserplugin, which attempts to parse the sample data and corresponding answers automatically from the puzzle prose.An aocd example parser plugin is anentry-pointin the"adventofcode.examples"group. It must be a callable accepting two arguments, like this:fromaocd.examplesimportExamplefromaocd.examplesimportPagedefmy_plugin(page:Page,datas:list[str])->list[Example]:"""my example parser implementation"""...# given the puzzle html found in "page", and a list of real user inputs found in# "datas" for potential comparison, the plugin function should scrape and return# a list of Example instances. Note that "datas" might be an empty list, if aocd# doesn't have any real user inputs cached locally.returnresultThis callable must return a list ofExampleinstances. If no examples can be parsed, you should return an empty list[], rather thanNone.Any package providing an example parser should register an entry point in the"adventofcode.examples"group within the package metadata, by adding something like this in yourpyproject.tomlfile:[project.entry-points."adventofcode.examples"]my_example_parser="my_module:my_plugin"See theEntry pointssection inPEP 621 – Storing project metadata in pyproject.tomlfor more information.How to use/select a parser plugin?When an example parser package is correctly installed alongsideadvent-of-code-data >= 2.0.0, it will show up as a choice inaoce --help:$aoce--help
usage:aoce[-h][-e{reference,simple}][-y2015+[2015+...]][-v]options:-h,--helpshowthishelpmessageandexit-e{reference,simple},--example-parser{reference,simple}plugintouseforexampleextractiontesting(default:reference)-y2015+[2015+...],--years2015+[2015+...]yearstoruntheparseragainst(canspecifymultiple)-v,--verboseincreasedlogging(-vINFO,-vvDEBUG)And it should be selectable for use/verification with$aoce--example-parser=my_example_parserTo print the actual results produced by a parser against a single puzzle, you may useaocd --example-parser. To demonstrate using the results for--- Day 1: Calorie Counting ---again:$aocd20221--example-parser=reference---Day1:CalorieCounting---https://adventofcode.com/2022/day/1
-------------------------------Exampledata1/1-------------------------------10002000300040005000600070008000900010000--------------------------------------------------------------------------------
answer_a:24000answer_b:45000--------------------------------------------------------------------------------Why a plugin? Wouldn't it be simpler to write a parser in aocd directly?I've created this package, and the corresponding testeraoceinadvent-of-code-data, to open it up to the community to try and come up with a better parser.
This package exemplifies the interface that a parser should work with, so to speak, andaocduses this plugin fordogfooding. As an added benefit, it means the example parsing can be frequently updated to ensure correct results are returned for previous puzzles, without requiring a new release ofaocditself.There are so many creative and smart people hacking on AoC that I'm sure someone can up with something much better than I was able to!
The default implementation fromaocdfails more than 40% of the time, so you don't have a very high bar to beat. Anything that scores better thanaoce -e simpleis an improvement. If someone comes up with a better-performing parser than "take the first pre as input data, take answers from the last codeblocks in each article", I will make their implementation the new default in a future version ofaocd.If you're considering writing an example parser, it's not advisable to strive for 100% success rate, that will be super-difficult if notimpossible.
Some of the puzzles havemany examples, some arereally tricky to parse, and some offerno example at all.
But difficulty aside, the main reason is that you needn't "overfit" to previous puzzles.
This is becauseadvent-of-code-dataalways intends to return correct example datafor past puzzlesby hardcoding therelevant code-block locations. The package uses aCalVer-ish scheme where the version string indicates up until which date the results have been manually verified. So,aocd-example-parserversion 2023.12.17 can be expected to return verified results for all Advent of Code puzzles <= 2023/12/17, and an unverified best-effort result for 2023/12/18+.The only thing that matters for a parser plugin is how well it can perform for a never-before seen puzzle.
That is, the goal is to maximize theprobabilitythat your parser will somehow find the right result at the instant a new puzzle unlocks.How well does the default implementation perform?It depends on the year.
The locations got a lot more consistent in recent years, so it has performed better more recently.
The final line thataocescript prints out is a rough percentage the parser got right.$forYEARin{2015..2022};doecho-n"$YEAR"&&aoce-esimple-y$YEAR|tail-1;done2015plugin'simple'scored78/336(23.2%)2016plugin'simple'scored53/159(33.3%)2017plugin'simple'scored85/221(38.5%)2018plugin'simple'scored69/212(32.5%)2019plugin'simple'scored43/204(21.1%)2020plugin'simple'scored67/183(36.6%)2021plugin'simple'scored82/152(53.9%)2022plugin'simple'scored71/120(59.2%)Averaging across all years, we're currently right about a third of the time:$aoce-esimple|tail-1
plugin'simple'scored548/1587(34.5%)Of course, the "reference" plugin is always correctfor historical puzzles.$aoce-ereference|tail-1
plugin'reference'scored1587/1587(100.0%)