Package: dali-rp2
DaLI for RP2 v0.6.9Privacy-focused, free, extensible data loader for RP2 (the crypto tax calculator)Table of ContentsIntroductionLicenseDownloadInstallationUbuntu LinuxmacOSWindows 10Other Unix-like SystemsRunningLinux, macOS, Windows 10 and Other Unix-like SystemsWindows 10Configuration FileReporting BugsContributingDeveloper DocumentationFrequently Asked QuestionsChange LogIntroductionDaLI(Data Loader Interface) is a data loader and input generator forRP2, the privacy-focused, free, non-commercial, open-source, community-driven multi-country cryptocurrency tax calculator: DaLI removes the need to manually prepare RP2 input files. Just like RP2, DaLI is also free, non-commercial, open-source and it prioritizes user privacy by storing crypto transaction data on the user's computer and never sending it anywhere else.It performs the following operations:it reads in crypto transaction information from multiples native sources: CSV files and/or REST-based services;it analyzes, processes and merges this data;it uses the processed data to generate an ODS input file for RP2 and its respective configuration file.DaLI has a programmable plugin architecture fordata loaders(both CSV and REST-based),pair convertersandcountries. While some exchanges and wallets are already supported out-of-the-box, more are needed: help us make DaLI a robust open-source, community-driven crypto data loader bycontributingplugins for exchanges and wallets! Checkopen issuesor open a new one.DaLI hasunit testcoverage to reduce the risk of regression.Note that DaLI has RP2 as a dependency, so installing DaLI causes RP2 to be installed as well.IMPORTANT DISCLAIMER:DaLI offers no guarantee of correctness (read thelicense): always verify results with the help of a tax professional.The author of DaLI and RP2 is not a tax professional, but has used RP2 and DaLI personally for a few years.LicenseDaLI is released under the terms of Apache License Version 2.0. For more information seeLICENSEorhttp://www.apache.org/licenses/LICENSE-2.0.DownloadThe latest version of DaLI can be downloaded at:https://pypi.org/project/dali-rp2/InstallationDaLI has been tested on Ubuntu Linux, macOS and Windows 10 but it should work on all systems that have Python version 3.7.0 or greater.Installation on Ubuntu LinuxOpen a terminal window and enter the following commands:sudo apt-get updatesudo apt-get install python3 python3-pipThen install DaLI:pip install dali-rp2Installation on macOSFirst make sureHomebrewis installed, then open a terminal window and enter the following commands:brew updatebrew install python3Then install DaLI:pip install dali-rp2Installation on Windows 10First make surePython3.7 or greater is installed (in the Python installer window be sure to click on "Add Python to PATH"), then open a PowerShell window and enter the following:pip install dali-rp2Installation on Other Unix-like Systemsinstall python 3.7 or greaterinstall pip3pip install dali-rp2RunningDaLI reads in a user-prepared configuration file inINI format, which is used to initialize data loaders and configure DaLI's behavior. The format of the configuration file is described in detail in theconfiguration filedocumentation.An example of a configuration file can be found intest_config.ini.After processing the configuration file, DaLI reads crypto data from native sources and generates a RP2 input ODS file and a RP2 configuration file in theoutputdirectory or where specified with the-oCLI option.The DaLI executable is country-dependent:dali_<country_code>, where country code is aISO 3166-1 alpha-2, 2-letter identifier (e.g.dali_us,dali_jp, etc).ProvidedDaLI is installed, to generate RP2 input files from the example configuration, open a terminal window (or PowerShell if on Windows).Create the directory structure and download the example input files using wget or manually.mkdir -p inputIf your system has wget:wget https://raw.githubusercontent.com/eprbell/dali-rp2/main/config/test_config.iniwget https://raw.githubusercontent.com/eprbell/dali-rp2/main/input/test_manual_in.csv -P input/wget https://raw.githubusercontent.com/eprbell/dali-rp2/main/input/test_manual_intra.csv -P input/wget https://raw.githubusercontent.com/eprbell/dali-rp2/main/input/test_trezor_alice_btc.csv -P input/wget https://raw.githubusercontent.com/eprbell/dali-rp2/main/input/test_trezor_bob_btc.csv -P input/If your system doesn't have wget, download the following files manually:test_config.initest_manual_in.csvtest_manual_intra.csvtest_trezor_alice.csvtest_trezor_bob.csvAnd move the csv files to the input directory:mv *.csv inputThen generate the US output files for rp2:dali_us -s -o output -p test_ test_config.ini-soption allows DaLI to retrieve spot price information from the Internet, when it's not available from the CSV files or REST endpoints.-ooption specifies the directory where the ODS output file is generated.-pis the prefix of the output file.test_config.iniis the configuration that ties the inputs together.To print command usage information for thedali_uscommand:dali_us --helpFinally compute taxes with RP2 using the generated input files (using both FIFO and LIFO accounting methods):rp2_us -m fifo -o output/ -p rp2_ output/test_crypto_data.config output/test_crypto_data.odsrp2_us -m lifo -o output/ -p rp2_ output/test_crypto_data.config output/test_crypto_data.odsConfiguration FileRead theconfiguration filedocumentation.Reporting BugsRead theContributingdocument.ContributingRead theContributingdocument.Developer DocumentationRead thedeveloper documentation.Frequently Asked QuestionsRead theuser FAQ listand thedeveloper FAQ list.Change LogRead theChange Logdocument.RP2 Change Log0.6.9new Dijkstra-based pair converter implementation (#142)added Coinbase Pro-based CCXT pair converter (#150)fixed a bug in cache setup of CCXT pair converters (#149)fixed a bug in Bitbank fee-only transactions (#155)fixed spreadsheet size, which was set to a large value regardless of how many rows it contained (#147)added XYMUSDT alternative market to CCXT pair converter (#158)various optimizations (#151, #152))various documentation fixes (#144, #148)0.6.8Coinbase data loader: added a new flavor of transfer transaction that was slightly different from the other ones (#137)0.6.7Coinbase data loader: fixed bug in advanced trade buy logic was not accounting for fee correctlyCoinbase data loader: added support for moving coins into staking pool (#132)0.6.5new Trezor CSV plugin supporting the latest format from Trezor Suite0.6.4added new RP2 Input data loader plugin, which can read RP2 input format data back into DaLIimprovements to Kraken CSV pair converter plugin: price data files are now downloaded locally and chunked to avoid excessive memory useimprovements to Coinbase plugin: added support for advanced_trade_fill, cardspend and cardbuybackvarious smaller fixes and improvements0.6.3added CCXT pair converter subclasses that are fixed to one exchange (Kraken, Binance)added fiat deposit support to Bitbank pluginsmall fixes and improvements0.6.2added Kraken CSV price importervarious fixes to CCXT-based pluginssmall improvements to some CSV data loadersupdated documentation0.6.1added CCXT-based abstract superclass for REST data loader plugins (#80): this makes it much easier to add a new REST-based data loaderadded Bitbank REST input plugin and CSV supplemental input pluginadded Coincheck CSV supplemental input pluginupdated RP2 to 1.3.1updated documentation0.6.0As of RP2 1.3.0, RP2 configuration files are no longer expressed in JSON format: they now use the INI format. So DaLI has been updated to generates INI-format configuration files instead of JSON. Any old JSON-format configuration file can be converted to the new INI format with the following command: rp2_config <json_config>added Pionex CSV data loader plugin0.5.2added Binance.com REST data loader pluginadded Binance.com supplemental (CSV) data loader plugin (for autoinvest purchases and ETH to BETH conversions)added CCXT-based pair converterupdated documentation0.5.1updated RP2 dependency to latest version, which fixes 2 bugs in HIFO. If you're using HIFO be sure to use RP2 1.10 or better.tweaked documentation0.5.0add top-level multi-thread support: it's now possible to run data loader plugins in parallel, using the -t option (which selects the number of parallel threads)added Nexo CSV input pluginCoinbase REST input plugin: added support for Coinbase Earn Reversals (due to CC refunds, etc.)BlockFI CSV input plugin: added new transaction typesrevised and improved all documentationadded FAQs0.4.12added country plugin infrastructure (US is the default country plugin). Default fiat is no longer hardcoded to USD (it now comes from the country plugin)updated generators and rest of the code to use the native fiat from the country plugin (previously it was hard coded as USD)the dali script has been renamed to dali_us to support the new country plugin architectureadded exchange hint to pair converter APIadded Ledger CSV data loaderadded credit card spend transactions to Coinbase pluginrenamed cache folder from .cache to .dali_cacheupdated documentation to reflect latest changessmall additions and fixes to documentation and code0.4.9added new pair conversion plugin infrastructureadded support for currency conversion to transaction resolver (useful to support non-USD fiats from foreign exchanges)fixed PR #44: added configurable historical data behavior and improved cachingsmall improvements to code and documentation0.4.8fixed PR #35: added exchange_withdrawal transaction type to Coinbase pluginvarious documentation improvementsupdated RP2 dependency to 1.0.00.4.7fixed issue #34: sometimes Coinbase returns bad fiat data for crypto conversions (fiat amount of sale < fiat amount of buy). DaLI used to raise an exception: now it issues a warning and sets the fiat fee to 0Coinbase plugin: _process_transfer() refactoring0.4.6fixed PRs #31 and #36: various improvements to the logic that reads prices from Web0.4.5Coinbase Pro: fixed a bug which caused the total fiat value (including fee) to be incorrect for certain crypto conversionsadded developer FAQ0.4.4fixed a regression introduced recently: in certain cases, some fees related to conversions would no longer be generated (thus causing small deductions to be ignored)added support for USDC: previously it was conflated into fiat and didn't generate a tab, now it doesCoinbase plugin: added support for fiat_withdrawal and fiat_deposit transactions. They are now tracked internally and no longer cause an unsupported transaction warningCoinbase Pro plugin: added support for "conversion" native transactions (these seem to occur when CBPro converts from/to a stable coin)minor changes to documentation0.4.3rewrote Coinbase coin swap logic: it now handles correctly swaps from stable coins with a fee (previously it would ignore the fee)various documentation fixes and some refactoring0.4.2fixed issue #21: fee-typed out-transactions in the manual plugin would cause an exceptionreviewed all documentation0.4.1minor documentation fixes0.4.0fixed issue #20: Coinbase plugin now supports the staking_reward transaction typefix for Coinbase reporting fiat with low precision (only 2 decimal digits): if the value is less than 1c Coinbase rounds it to zero, which caused various computation problems (spot_price, etc.). As a workaround, when this condition is detected the plugin sets affected fields to UNKNOWN or None (depending on their nature), so that they can be filled later by the transaction resolver and RP20.3.28added threading support to Coinbase Prothread count can now be specified in the plugin configuration section of the .ini fileadditional fix in missing fiat_fee issue in crypto swaps on Coinbase (issue #15)updated documentation0.3.27fix for missing fiat_fee issue in crypto swaps on Coinbase (issue #15)0.3.26-s CLI option to read missing spot prices from Web: rewrote implementation from Yahoo Finance (daily granularity) to Coinbase Pro (minute granularity)moved cache logic to its own module, so it can be used in multiple places: plugins, transaction_resolver, etc.fixed issue #19: the user needs to pass -s to read highly granular price data from Coinbase Pro, when it's missing (or computed as 0)0.3.25merged PR #18: Coinbase plugin wasn't handling crypto conversions correctly and generated negative amounts.0.3.23merged PR #14: in Coinbase Pro plugin crypto fee is now modeled correctly both for buy and sell-side fillsfixed issue #16: added support for inflation_reward transaction in Coinbase pluginadded load caching to speed up development (PR #12)added parallelism to Coinbase plugin (PR #12)0.3.22fixed issue #16: added support for Coinbase inflation reward transactionadded error log for unknown transaction type in all plugins, so they are easy to identify and reportadded FAQ on supported exchanges/walletsadded support for more transaction types in BlockFi plugin (PR #11)0.3.21fixed bug reported in issuehttps://github.com/eprbell/dali-rp2/issues/10fixed from/to_currency detection in Coinbase Pro plugin (PR #9)added initial version of developer FAQ document:https://github.com/eprbell/dali-rp2/blob/main/docs/developer_faq.mdminor fixes to documentation0.3.20fixed a bug in transaction resolver: IN -> IN transaction hint was assigning wrong value to fiat_feefixed a bug in Coinbase plugin: receiving a crypto gift from another Coinbase user generated a transaction with wrong fiat_in_with_fee valuereworked some of the code to use latest RP2 features: specifically fee-only transactions and crypto_fee in in-transactions. E.g.: Coinbase Pro coin swap code, transaction resolver INTRA -> OUT transaction hint application, etc.DaLI output file (the RP2 input file) now has in-transaction crypto_fee. Also updated unit test golden filerefactored all transaction constructor calls to use keyword arg calling styleupdated setup.cfg dependencies to latest RP20.3.19Trezor plugin: AM/PM was not parsed correctly in timestamp. Fixed.Minor fixes0.3.18fixed a limitation in timestamp processing in Trezor and Trezor Old plugins. The timezone that was passed to the plugin constructor could only be an ISO 8601 format offset: something like "America/Los Angeles" would not be accepted. This caused problems with daylight saving timestamps in Trezor CSV files: daylight savings time would just be ignored, thus causing Trezor summer timestamps to be 1 hour offupdated input, input/golden, config and docs/configuration_file.md to reflect the above fixadded pytz stubsv0.3.17added new FAQ on adding support for new wallets and exchangesminor fixesv0.3.16Trezor plugin: dusting attack was incorrectly interpreted as cost-only transaction. Now it issues a warning to the userminor fixes to documentationv0.3.15minor edits to documentation and metafilesv0.3.12fixed a few broken links in README.md (when used from Pypi only)fixed CLI --help descriptionminor improvements to documentationv0.3.7major improvements to documentationadded notes in generated file to describe crypto conversion transactions in Coinbase Prosmall performance improvement: signature reflexive call is now called only once per class, instead of once per instancev0.3.6template.ods was missing from the final package. Fixed in setup.cfgvarious fixes to documentationv0.3.4First version tracked in change log and uploaded to Github