Package: beanahead
beanaheadBeanahead administers future transactions forbeancountledgers.It let's you:Generate regular expected transactions.Define ad hoc expected transactions.Reconcile expected transactions against imported transactions.It's useful to:Forecast account balances based on expected income and payments.Add transactions between updating accounts.Installation$ pip install beanaheadThe only direct dependencies arebeancountandpandas(pandas could be made optional, see#1).:information_source: Thebeancountrequirement is beancount v2. It's intended thatbeanaheadwill be upgraded to support beancount v3 (currently in development) when v3 is completed and published to PyPI.Really briefly, how it worksExpected transactions are defined on separate ledgers.Newly imported entries are reconciled against expected transactions before being included to the main ledger (beanahead does not need to touch your main ledger).Briefly, how it worksRegular Expected Transactions (electicity bill, rent, etc) are defined by including a single transaction to a dedicated beancount file, by convention 'rx_def.beancount'.Theaddrxcommand is used to populate a Regular Expected Transactions ledger with transactions. This ledger, by convention 'rx.beancount', is 'included' to your main .beancount ledger.ad hoc Expected Transactions are added to the Expected Transactions ledger, by convention 'x.beancount'. This ledger is also 'included' to your main .beancount ledger.Thereconcomamnd offers a cli to reconcile newly imported transactions with transactions on the expected transactions ledgers.Newly imported transactions are updated to reflect any missing narration, tags, meta and 'other side' postings defined on the corresponding expected transactions.Reconciled expected transactions are removed from their respective ledgers.In keeping with thebeancountspirit,beanaheadprovides all its functionality via a CLI (the underlying functions are available within the codebase should you prefer). There are five commands, all subcommands ofbeanahead.$ beanahead --help
usage: beanahead [-h] [--version] {make,addrx,recon,exp,inject} ...

subcommands:
  {make,addrx,recon,exp,inject}
    make                make a new beanahead file.
    addrx               add Regular Expected Transactions.
    recon               reconcile new transactions.
    exp                 administer expired expected transactions.
    inject              inject new transactions.:information_source: Wherever this README shows the output from a--helpoption, only an abridged version of the output is shown. Run the command at the command line for the full help, e.g.beanahead --help. For subcommands the help includes the documentation of the underlying function.Using beanaheadIndexMaking beanahead filesRegular Expected TransactionsDefining regular transactionsfreqPostingsrollfinalUpdating definitionsAdding regular transactionsad hoc transactionsDefining the payeeReconcilingMatchingUpdatingInjectionExpired expected transactionsWorth rememberingAlternative packagesbeancount recommendationsLicenceMaking beanahead filesFirst off, you'll need to use themakecommand to create a new.beancountfile or three.:information_source: You'll probably want these files to be in the same directory as your exisitng main ledger. The following examples assume you want to create the files in the current working directory. Pass the --dirpath (or -d) option to specify a different directory.If you want to include Regular Expected Transactions you'll need to create a definitions file and a dedicated ledger....$ beanahead make rx_def -f rx_def
$ beanahead make rx -f rx...and add the following line towards the top of your main ledger.include "rx.beancount"If you want to include ad hoc Expected Transactions you'll need to create a separate dedicated ledger...$ beanahead make x -f x...and add the following line towards the top of your main ledger.include "x.beancount"So, if you want to include both regular and ad hoc expected transactions then you should have created three new.beancountfiles and added two 'include' lines to top of your main ledger.:information_source: The -f option provides for defining the filename (makewill add the.beancountextension). If -f is not passed then default names will be used which are as those explicitly passed in the examples.:information_source: The quoted file in the 'include' lines should reflect the path to the expected transactions ledger from the directory in which your main ledger is located. The above examples assume that the expected transactions ledgers have the default filenames and are located in the same directory as the main ledger.Theexamples/new_empty_filesfolder includes a sample of each of the above newly created files.Regular Expected Transaction Definitions fileRegular Expected Transactions LedgerExpected Transactions LedgerRegular Expected TransactionsRegular expected transactions are defined on the Regular Expected TransactionDefinitionsfile. Theaddrxcommand can then be used to populate the Regular Expected TransactionsLedgerwith transactions generated from these definitions.Defining regular transactionsA new regular transaction can be defined at any time by adding a single transaction to the definitions file (the 'initial definition'). The date of this transaction will serve as the anchor off which future transactions are evaluated.The following initial definition would generate regular transactions on the 5th of every month, with the first generated transaction dated 2022-10-05.2022-10-05 * "EDISON" "Electricity, monthly fixed tarrif"
  freq: "m"
  Assets:US:BofA:Checking                     -65.00 USD
  Expenses:Home:ElectricityEach initial definition must:have auniquepayee.include thefreqmeta field to define the transaction frequency.include at least twopostingsof which at least one should be to a balance sheet account ("Assets" or "Liabilities").Initial definitions can optionally include:therollmeta field to define if transactions falling on weekends should roll forwards.thefinalmeta field to define the last transaction date.additionalpostings.custom meta fields.tagsrx_defs.beancountoffers an example of a new definitions file before any transactions have been generated. The initial definitions there cover a variety of circumstances, based loosely on selected sampling of beancount'sexample.beancountledger.freqThefreqmeta field is used to specify the frequency with which regular transactions should be generated.Asimple frequencycan be specified as "w", "m" and "y", respectively for weekly, monthly and yearly. The unit can be prefixed with a value to specify a multiple, for example "2w" for fortnightly or "3m" for quarterly.Alternatively, the frequency can be specified with apandas frequency. For example "BAS-MAR" defines the frequency as the first business day of every March.PostingsEach definition must include a posting to an account which the regular transactions will appear on the statements of. This can be an "Assets" account (for example, for Direct Debits) or a "Liabilities" account (for example, for regular charges to a credit card). If the amount is variable then just stick in an estimate or the amount that you wish to budget for.At least one posting to an account on the 'other side' must be included (e.g. to an "Expenses" account). Any number of other postings can be included.When the regular expected transactions are later reconciled with transactions imported from your statements, thereconcommand will update the imported transactions with these 'other-side' postings (seereconciling).:information_source: If the transaction is balanced with only one 'other-side' posting then you're better off not including an amount for it. (By including an amount, if the imported transaction has a different amount then you'll need to manually amend the posting on the updated imported transaction.):information_source: If the transaction is split between various 'other-side' postings then it will be necessary to define an amount for at least all but one of these. In this case, if the imported transaction value differs from the expected transaction value then you may want to revise the amounts of those postings for which estimates were included.RollBy default, any generated transaction that would be dated on a weekend will be rolled forward to the following Monday. This behaviour can be overriden by the 'roll' meta field.2022-11-13 * "ETrade Transfer" "Transfering accumulated savings to other account"
  freq: "3m"
  roll: FALSE
  Assets:US:BofA:Checking                       -4000 USD
  Assets:US:ETrade:CashThe above initial definition is dated 2022-11-13, which is a Sunday. By specifying roll as FALSE the first transaction generated will be dated 2022-11-13 regardless that this is a Sunday. Thereafter transactions will be defined on the 13th of each month regardless of whether these dates represent weekdays or weekends.:warning: the roll field's value must be in captials and NOT quoted.:information_source: initial definitions should always be dated on the 'usual' payment day even if that falls on a weekend. For example...2022-10-16 * "Verizon" "Telecoms, monthly variable"
freq: "m"
Assets:US:BofA:Checking                       -55.00 USD
Expenses:Home:PhoneThis initial definition is dated 2022-10-16 which is a Sunday. The first generated transaction will be automatcially rolled forward to 2022-10-17. All transactions thereafter will be dated as the 16th of each month whenever the 16th is a weekday or otherwise rolled forward to the following Monday.FinalThe 'final' meta field can be used to define a final transaction date. No transactions will be generated that would be dated later than this date (as evaluated prior to any rolling).2022-10-31 * "Chase" "Chase Hire Purchase"
  freq: "BM"
  final: 2022-11-30
  Liabilities:US:Chase:HirePurchase                322.00 USD
  Assets:US:BofA:Checking:warning: the final field's value should NOT be quoted.A definition will be automaticaly removed from the definitions file after any final transaction has been generated.Updating definitionsTheaddrxcommand updates the definitions file whenever the ledger is populated with new transactions:any definition for which a new transaction was generated will be updated to reflect the transaction that would immediately follow the last transaction that was added to the ledger.Defintions are grouped by balance sheet account* and type of account(s) of the 'other side', for example 'Income', 'Expenses' etc. Each group is introduced with a title row. Within each group definitions are sorted by payee. (*The balance sheet account is assumed as the first "Assets" or "Liabilities" account defined in the postings.):information_source: New definitions can be added at any time anywhwere under the line;; Enter definitions after this line...- they'll find their way to the corresponding section when the file is next updated.rx_def_updated.beancountis the updatedrx_def.beancountfile after adding regular transactions to the ledger through to 2022-12-31. :warning: Notice that all comments are lost when a definitions file is updated.Adding regular transactionsTheaddrxcommand is used to populate a Regular Expected Transactions Ledger with transactions evaluated from a definitions file.$ beanahead addrx --help
usage: beanahead addrx [-h] [-e] defs rx-ledger main-ledger

positional arguments:
  defs         path to Regular Expected Transactions Definition file.
  rx-ledger    path to Regular Expected Transactions Ledger file.
  main-ledger  path to main Ledger file.

optional arguments:
  -h, --help   show this help message and exit
  -e , --end   date to which to create new transactions, iso format,e.g. '2020-09-30'. Default 2023-01-11.For example...$ beanahead addrx rx_def rx ledger -e 2022-12-31The above command:Gets the definitions from the filerx_def.beancountin the current working directory.For each definition, generates transactions from the defintion date through to '2022-12-31' (both inclusive).Adds transactions to therx.beancountledger.Updates therx_def.beancountfile (asUpdating definitions).Verifies that the main ledger,ledger.beancount, loads without errors. The path to the main ledger is passed as the third positional argument (this ledger should be the main ledger to which the 'insert "rx.beancount"' line was added).addrxrequires this file only to verify that no errors have arisen as a result of introducing the new transactions to the Regular Expected Transactions Ledger.In the event the main ledger loads with errors then changes made to the definitions file and Regular Expected Transactions Ledger are reverted and advices printed.If the command is executed as above with the files in theexamples/defsfolder then the empty rx ledger there will be populated with transactions. The rx ledger would end up asrx_updated.beancountwhilst the definitions file would be updated asrx_def_updated.beancount.ad hoc transactionsCreating ad hoc expected transactions is as simple as adding transactions to an Expected Transactions Ledger created via$ beanahead make x <filename>. Thex.beancountfile offers an example (again, loosely based on selected sampling of beancount'sexample.beancountledger).Jotting down ad hoc transactions is useful to record transaction details 'in the moment' when you have in mind the 'other-side' postings and maybe know the narration or tag that you might forget by the time you next get round to downloading statements and updating your main ledger.Transactions can be listed on the ledger in any order. Whenever a transaction on the ledger is reconciled with an imported transaction (extracted from a statement) the ledger is rewritten and any remaining transactions are reordered.:warning: Any comments will be lost whenever an Expected Transactions Ledger is rewritten.Defining thepayeeThe reconciliation of imported transactions with expected transactions (reconciling) can be greatly aided by judiciously naming the expected transactions payee.Beanahead will treat each 'word' defined in the expected payee as a separate string. An expected payee will match with the payee of any new transaction that includesanyof those strings. So:Do not include short words that represent common syllables.
For example, "Top of the World" will match with "The corner shop", "Another Day" and "Super Offers".Use few unambiguous words. For example "Top World". Even just "Top" might be a better choice.:information_source: The transaction that ends up on your main ledger will have the payee as defined on the statement, not "Top"!
:information_source: matches are case-insensitiveReconcilingThereconcommand provides for reconciling imported transactions with expected transactions.$ beanahead recon --help
usage: beanahead recon [-h] [-o OUTPUT] [-k] [-r] new ledgers [ledgers ...]

positional arguments:
  new                   path to new transactions beancount file. Can be
                        absolute or relative to the current working directory.
  ledgers               paths to one or more Regular Expected Transactions
                        Ledgers against which to reconcile incoming
                        transactions.

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        path to which to write updated new entries. Can be
                        absolute or relative to the current working directory.
                        By default will overwrite 'incoming'.
  -k, --keep, --no-remove
                        flag to not remove reconciled transactions from ledgers.
                        (By default reconciled transactions will be removed.)
  -r, --reverse         flag to write updated entries in descending order. By
                        default will be written in ascending order.For example...$ beanahead recon extraction rx x -o injectionThe above command:Gets beancount entries from the fileextraction.beancountlocated in the current working directory. This file should contain only new entries as returned by the beancountextractcommand and destined for your beancount ledger.Gets the expected transactions from the ledger filesrx.beancountandx.beancount, also located in the current working directory.Matchesexpected transactions with new transactions.The user is requested to confirm or reject each propsosed match.Where an expected transaction matches more than one new transaction, the user is requested to select which of the new transactions to match (or none).Updatesmatched new transactions with information gleaned from the corresponding expected transaction.Outputs the updated entries. By default overwrites the file passed as thenewargument (extraction.beancountabove). Alternatively, a path can be passed to the--outputoption, as in this case where output will be written toinjection.beancountin the current working directory.Entries are by default sorted in ascending order. Pass the-rflag or--reverseoption to sort in descending order.Removes reconciled expected transactions from their respective ledgers. (The-kflag can be passed to 'keep' the expected transactions.)If the command is executed as above with the files in theexamples/reconfolder and the matches are confirmed then:aninjection.beancountfile will be created that looks likethis.reconciled transactions will be removed fromrx.beancount, such that it ends up likethis.reconciled transactions will be removed fromx.beancount, such that it ends up likethis.MatchingBeanahead matches expected with imported transactions based on:payee (seeDefining the payee)similarity in accountscloseness of datescloseness of amountsAs a bare minimum, matches require that the new and expected transactions:are dated within 5 days of each otherinclude a posting to the same Asset accounteither have matching payee or the imported transaction amount is no more than 2% different from the expectedIf you want a better insight into how matches are evaluated, check out thereconcilemodule to get under-the-bonnet.UpdatingBeanahead will update the following fields of matched imported transactions to include any values specifed for the corresponding expected transaction. Any existing values on the imported transaction willnotbe overwritten.narrationtags(excluding #rx_txn and #x_txn)meta(excluding beanahead meta fields such as 'final', 'roll' etc)postingspostings on the expected transactions will be added to the imported transaction if the imported transaction does not otherwise include a posting to the corresponding account.if the imported and expected transactions include postings to the same account and only the expected transaction defines a number, the imported transaction's posting will be updated to reflect the value as defined on the expected transaction.InjectionThe output fromreconcan be copied directly into your main ledger. If you're happy to append the full contents 'as is' to the end of your ledger then theinjectcommand will do it for you.$ beanahead inject --help
usage: beanahead inject [-h] injection ledger

positional arguments:
  injection   path to beancount file containing the new entries to be
              injected. Can be absolute or relative to the current
              working directory.
  ledger      path to beancount ledger to which new entires are to
              be appended. Can be absolute or relative to the current
              working directory.So...$ beanahead inject injection my_ledger...would append the updated entires in theinjection.beancountfile to the end of themy_ledger.beancountfile (both files in the current working directory).Expired expected transactionsNow that your main ledger has been updated with the new entries it'll be necessary tobean-checkit to see if all's well. Chances are you'll have to enter some manual postings to balance some transactions.It's also possible that your balance checks are failing because some expected transactions were included in the new entries although weren't matched (and so now are duplicated), or simply didn't come in (that credit you were waiting for). Beanahead can't chase your debtors but theexpcommand can at least deal with any expired expected transactions.$ beanahead exp --help
usage: beanahead exp [-h] ledgers [ledgers ...]

positional arguments:
  ledgers     paths to one or more Regular Expected Transactions Ledgers
              against which to administer expired transactions.For example:$ beanahead exp rx xThe above command:Gets the transactions on the expected transactions ledger files 'rx.beancount' and 'x.beancount' (both in the current working directory).Offers the user the following options for each transaction that is dated prior to 'today'.Move transaction forwards to 'tomorrow'.Move transaction forwards to a user-defined date.Remove transaction from ledger.Leave transaction as is.Rewrites the expected transactions ledgers (if applicable) to reflect the requested changes.With a bit of luck and perhaps a tweak or two to your ledger, yourbean-checkshould now be checking out.:information_source: An alternative to usingexpis to manually redate / remove transactions on the expected transactions ledgers.Worth remembering:warning: Whenever an expected transactions ledger or the regular expected transaction definition files are updated the entries are resorted and the file is overwritten - anything that is not a directive (e.g. comments) will be lost.Alternative packagesThe beancount community offers a considerable array of add-on packages, many of which are well-rated and maintained. Below I've noted those I know of with functionality that includes some of whatbeanaheadoffers. Which package you're likely to find most useful will come down to your specific circumstances and requirements - horses for courses.beancount-import- an importer interface. Functionality provides for adding expected transactions directly to the main ledger and later merging these with imported transactions via a web-based UI. It requires implementing the importer interface and doesn't directly provide for regular expected transactions. But, if that import interface works for you then you'll probably want to be usingbeancount-import. (If you need the regular trasactions functionality provided bybeanahead, just usebeanaheadto generate the transactions, copy them over to your ledger and letbeancount-importhandle the subsequent reconcilation.)If you can recommend any other alternative package please raise a PR to add it here.beancountrecommendationsIf you don't already, it worth trying out a beancount syntax-highlighter extension. Have a look at the'Editor Support' section of the Awesome Beancountrepo to see if there's one available for your prefered editor.More broadly,awesome-beancountis a great refrence for all things beancount. FWIW, these are my most-awesome of awesome-beancount:Siddharnt Goel's blog postusesbeancountexamples to explain Plain Text Accounting and double-entry accounting. A great introduction to these concepts.Thebeancount mailing listis the place to look for answers and raise questions.reds-rantsoffers a comprehensive set of blogs on everything importing. It's tremendous.smart_importeroffers import hooks that include one for auto-completion of postings based on machine-learning trained on past entries. Smart stuff indeed.beancount-importis an importer interface. If it works for you to be locked into an importer interface, this is a great one to be locked into.LicenseMIT License