Package: ledger-autosync
ledger-autosync is a program to pull down transactions from your bank
and createledgertransactions for them.
It is designed to only create transactions that are not already
present in your ledger files (that is, it will deduplicate
transactions). This should make it comparable to some of the automated
synchronization features available in products like GnuCash, Mint,
etc. In fact, ledger-autosync performs OFX import and synchronization
better than all the alternatives I have seen.Newsv1.0.0Versions of ledger-autosync before 1.0.0 printed the ofxid in a
slightly incorrect position. This should not effect usage of the
program, but if you would like to correct the error, see below for
more details.Featuressupportsledger3 andhledgerlike ledger, ledger-autosync will never modify your files directlyinteractive banking setup viaofxclientmultiple banks and accountssupport for non-US currenciessupport for 401k and investment accountstracks investments by share, not dollar valuesupport for complex transaction types, including transfers, buys,
sells, etc.import of downloaded OFX files, for banks not supporting automatic
downloadimport of downloaded CSV files from Paypal, Simple, Amazon and Mintany CSV file can be supported via pluginsPlatformsledger-autosync is developed on Linux with ledger 3 and python 3; it
has been tested on Windows (although it will run slower) and should
run on OS X. It requires ledger 3 or hledger, but it should run faster
with ledger, because it will not need to start a command to check
every transaction.QuickstartInstallationIf you are on Debian or Ubuntu, an (older) version of ledger-autosync
should be available for installation. Try:$ sudo apt-get install ledger-autosyncIf you use pip, you can install the latest released version:$ pip install ledger-autosyncYou can also install from source, if you have downloaded the source:$ python setup.py installYou may need to install the following libraries (on debian/ubuntu):$ sudo apt-get install libffi-dev libpython-dev libssl-dev libxml2-dev python-pip libxslt-devRunningOnce you have ledger-autosync installed, you can download an OFX file
from your bank and run ledger-autosync against it:$ ledger-autosync download.ofxThis should print a number of transactions to stdout. If you add these
transactions to your default ledger file (whatever is read when you
runledgerwithout arguments), you should find that if you run
ledger-autosync again, it should print no transactions. This is
because of the deduplicating feature: only new transactions will be
printed for insertion into your ledger files.Using the ofx protocol for automatic downloadledger-autosync also supports using the OFX protocol to automatically
connect to banks and download data. You can use the ofxclient program
(which should have been installed with ledger-autosync) to set up
banking:$ ofxclientWhen you have added your institution, quit ofxclient.(At least one user has reported being signed up for a pay service by
setting up OFX direct connect. Although this seems unusual, please be
aware of this.)Edit the generated~/ofxclient.inifile. Change thedescriptionfield of your accounts to the name used in ledger.
Optionally, move the~/ofxclient.inifile to your~/.configdirectory.Run:ledger-autosyncThis will download a maximum of 90 days previous activity from your
accounts. The output will be in ledger format and printed to stdout.
Add this output to your ledger file. When that is done, you can call:ledger-autosyncagain, and it should print nothing to stdout, because you already have
those transactions in your ledger.How it worksledger-autosync stores a unique identifier as metadata with each
transaction. (For OFX files, this is a unique ID provided by your
institution for each transaction.) When syncing with your bank, it
will check if the transaction exists by running the ledger or hledger
command. If the transaction exists, it does nothing. If it does not
exist, the transaction is printed to stdout.Payee matchingWhen generating transactions, ledger-autosync attempts to match
previous payees to determine the appropriate accounts. For instance,
if a previous payee wasGrocery store, and one posting was to the
accountExpenses:Food, ledger-autosync will use that account again.
If ledger-autosync can’t find a matching payee, it will useExpenses:Miscor the value of the--unknown-accountargument.The matching is not very sophisticated: it only does exact matching on
the payee, and it uses transaction with the matching payee. It is also
not currently working with CSV files.If you prefer to modify the payees to make them shorter than what is
generated by ledger-autosync, you can use theAutosyncPayeemetadata
field to indicate to ledger-autosync that it should use the longer
form for matching. For instance:2011/03/31 Grocery
  ; AutosyncPayee: Payment to Grocery store #12345 CALIFORNIA
  Assets:Bank                                  -$0.01
  Expenses:Food                                 $0.01would indicate to ledger-autosync that any payee with the namePayment to Grocery store #12345 CALIFORNIAshould use theExpenses:Foodaccount.ofxid/csvid metadata tagledger-autosync stores a metatag with every posting that it outputs to
support deduplication. This metadata tag is eitherofxid(for OFX
imports) orcsvidfor CSV imports.Pre-1.0.0 versions of ledger-autosync put this metadata tag in a
slightly incorrect place, associating the metadata tag with the
transaction itself, and not simply one posting. This should not effect
the usage of ledger-autosync, but if you would like to correct your
ledger files, there is a small python scriptfix_ofxid.pyincluded
with ledger-autosync. It can be run as:python fix_ofxid.py <input file>and will print a corrected file to stdout.Syncing a CSV fileIf you have a CSV file, you may also be able to import it using a
recent (installed via source) version of ledger-autosync.
ledger-autosync can currently process CSV files as provided by Paypal,
Amazon, or Mint. You can process the CSV file as follows:ledger-autosync /path/to/file.csv -a Assets:PaypalWith Amazon and Paypal CSV files, each row includes a unique
identifier, so ledger-autosync will be able to deduplicate against any
previously imported entries in your ledger files.With Mint, a unique identifier based on the data in the row is
generated and stored. If future downloads contain identical rows, they
will be deduplicated. This method is probably not as robust as a
method based on unique ids, but Mint does not provide a unique id, and
it should be better than nothing. It is likely to generate false
negatives: transactions that seem new, but are in fact old. It will
not generate false positives: transactions that are not generated
because they seem old.If you are a developer, you should fine it easy enough to add a new
CSV format to ledger-autosync. See, for example, theMintConverterclass in theledgerautosync/converter.pyfile in this repository.
See below for how to add these as plugins.AssertionsIf you supply the--assertionsflag, ledger-autosync will also
print out valid ledger assertions based on your bank balances at the
time of the sync. These otherwise empty transactions tell ledger that
your balanceshouldbe something at a given time, and if not, ledger
will fail with an error.401k and investment accountsIf you have a 401k account, ledger-autosync can help you to track the
state of it. You will need OFX files (or an OFX protocol connection as
set up by ofxclient) provided by your 401k.In general, your 401k account will consist of buy transactions,
transfers and reinvestments. The type will be printed in the payee
line after a colon (:)The buy transactions are your contributions to the 401k. These will be
printed as follows:2016/01/29 401k: buymf
  Assets:Retirement:401k                                 1.12345 FOOBAR @ $123.123456
  ; ofxid: 1234
  Income:Salary                                            -$138.32This means that you bought (contributed) $138.32 worth of FOOBAR (your
investment fund) at the price of $123.123456. The money to buy the
investment came from your income. In ledger-autosync, theAssets:Retirement:401kaccount is the one specified using the--accountcommand line, or configured in yourofxclient.ini.
TheIncome:Salaryis specified by the--unknown-accountoption.If the transaction is a “transfer” transaction, this usually means
either a fee or a change in your investment option:2014/06/30 401k: transfer: out
  Assets:Retirement:401k                                -1.61374 FOOBAR @ $123.123456
  ; ofxid: 1234
  Transfer                                                  $198.69You will need to examine your statements to determine if this was a
fee or a real transfer back into your 401k.Another type of transaction is a “reinvest” transaction:2014/06/30 401k: reinvest
  Assets:Retirement:401k                                0.060702 FOOBAR @ $123.123456
  ; ofxid: 1234
  Income:Interest                                            -$7.47This probably indicates a reinvestment of dividends. ledger-autosync
will printIncome:Interestas the other account.resyncBy default, ledger-autosync will process transactions backwards, and
stop when it sees a transaction that is already in ledger. To force it
to process all transactions up to the--maxdays back in time
(default: 90), use the--resyncoption. This can be useful when
increasing the--maxoption. For instance, if you previously
synchronized 90 days and now want to get 180 days of transactions,
ledger-autosync would stop before going back to 180 days without the--resyncoption.payee formatBy default, ledger-autosync attempts to generate a decent payee line
(the information that follows the date in a ledger transaction).
Unfortunately, because of differences in preference and in the format
of OFX files, it is not always possible to generate the user’s
preferred payee format. ledger-autosync supports apayee-formatoption that can be used to generate your preferred payee line. This
option is of the formatText {memo}, wherememois a
substitution based on the value of the transaction. Available
substitutions arememo,payee,txntype,accountandtferaction. For example:$ ledger-autosync --payee-format "Memo: {memo}"
2011/03/31 Memo: DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%This option is also available for CSV conversion. For CSV files, you
can substitution any of the values of the rows in the CSV file by
name. For instance, for Paypal files:$ ledger-autosync --payee-format "{Name} ({To Email Address})" -a Paypal paypal.csv
2016/06/04 Jane Doe (someone@example.net)python bindingsIf the ledger python bindings are available, ledger-autosync can use
them if you pass in the--pythonargument. Note, however, they can
be buggy, which is why they are disabled by defaultPlugin supportledger-autosync has support for plugins. By placing python files a
directory named~/.config/ledger-autosync/plugins/it should be
possible to automatically load python files from there. You may placeCsvCconvertersubclasses here, which will be selected based on the
columns in the CSV file being parsed and the FIELDSET of the CSV
converters. You may also place a singleOfxConverterin the plugin
directory, which will be used in place of the stockOfxConverter.Below is an example CSV converter, starting with the input CSV file:"Date","Name","Amount","Balance"
"11/30/2016","Dividend","$1.06","$1,000“The following converter in the file~/.config/ledger-autosync/plugins/my.py:from ledgerautosync.converter import CsvConverter, Posting, Transaction, Amount
import datetime
import re

class SomeConverter(CsvConverter):
    FIELDSET = set(["Date", "Name", Amount", "Balance"])

    def __init__(self, *args, **kwargs):
        super(SomeConverter, self).__init__(*args, **kwargs)

    def convert(self, row):
        md = re.match(r"^(\(?)\$([0-9,\.]+)", row['Amount'])
        amount = md.group(2).replace(",", "")
        if md.group(1) == "(":
            reverse = True
        else:
            reverse = False
        if reverse:
            account = 'expenses'
        else:
            account = 'income'
        return Transaction(
            date=datetime.datetime.strptime(row['Date'], "%m/%d/%Y"),
            payee=row['Name'],
            postings=[Posting(self.name, Amount(amount, '$', reverse=reverse)),
                      Posting(account, Amount(amount, '$', reverse=not(reverse)))])Runningledger-autosyncfile.csv-aassets:bankwill generate:2016/11/30 Dividend
    assets:bank                                $1.06
    income                                    -$1.06For more examples, seehttps://gitlab.com/egh/ledger-autosync/blob/master/ledgerautosync/converter.py#L421or theexample plugins directory.If you develop a converter that you think will be generally
useful, please consider submitting a pull request.Testingledger-autosync uses pytest for tests. To test, run pytest in the project directory. This will test the ledger, hledger and ledger-python interfaces. If hledger or the ledger-python interface is not found, these tests will be skipped.