Repository: rotki/rotki

Issue:
Profit and Loss Report - Received in Asset does not take fee of same type into account
When adding a trade that has a fee in the same asset e.g. BTC_EUR with a BTC fee. The Received in Asset field does not deduct the fee.

This also means that the amount without the fee subtracted is used for the BTC forks: BCH, BSV.

I could add the fee as a separate Ledger Action (it is a withdrawal fee), but the fork amounts in the report will still be incorrect. It's interesting these appear in the report before the forks happened but I guess this is ok as they're not claimed yet it's just to calculate cost?

Please advise the correct way to enter trades and withdrawal fees. Should the amount equal the amount bought on the exchange, or the amount received after withdrawal (less fee)? Should I put fees in the trade itself or a separate ledger action expense?




Discussion:
> It's interesting these appear in the report before the forks happened but I guess this is ok as they're not claimed yet it's just to calculate cost?

yes

> The Received in Asset field does not deduct the fee.

Fees are subtracted as an expense in their equivalent fiat value during accounting. It's probably not reduced if it's in the same asset as one of the two assets in the pair as you saw.

> Please advise the correct way to enter trades and withdrawal fees

Withdrawal fee? I am not sure I understand. What I understand as withdrawal fee is let's say how much binance would charge me to withdraw ETH. This we autodetect from the supported exchanges and handle properly.

As for trades, as I mentioned above the fees are considered as expense and are reduced from the profit of the trade in fiat value.  It's possible we don't reduce the asset in cost basis calculation for the fees. Which I would say if we don't it's something we should do and needs fixing. I will have to take a look to confirm later with a clear head.

If that is the case until it's fixed you can create a separate ledger entry, or wait for a fix of this issue.
I think it may be ok If the received in asset does not deduct the fee. It might be good to introduce an option that enables this per trade though.

The fees should definitely deduct from the cost basis. It would actually help if you exposed the running total balance for each asset so that each trade can be reconciled. If you allow to manually deposit fiat like EUR as well then you can also reconcile fiat balances too.

Resolving patch:
diff --git a/rotkehlchen/accounting/accountant.py b/rotkehlchen/accounting/accountant.py
--- a/rotkehlchen/accounting/accountant.py
+++ b/rotkehlchen/accounting/accountant.py
@@ -254,6 +254,8 @@ def trade_add_to_sell_events(self, trade: Trade, loan_settlement: bool) -> None:
                 receiving_amount=trade.amount * trade.rate,
                 gain_in_profit_currency=gain_in_profit_currency,
                 total_fee_in_profit_currency=fee_in_profit_currency,
+                fee_currency=trade.fee_currency,
+                fee_amount=trade.fee,
                 trade_rate=trade.rate,
                 rate_in_profit_currency=selling_rate,
                 timestamp=trade.timestamp,
@@ -267,6 +269,8 @@ def trade_add_to_sell_events(self, trade: Trade, loan_settlement: bool) -> None:
                 receiving_amount=None,
                 gain_in_profit_currency=gain_in_profit_currency,
                 total_fee_in_profit_currency=fee_in_profit_currency,
+                fee_currency=trade.fee_currency,
+                fee_amount=trade.fee,
                 rate_in_profit_currency=selling_rate,
                 timestamp=trade.timestamp,
                 loan_settlement=True,
@@ -619,6 +623,8 @@ def _process_action(
                 paid_with_asset=trade.quote_asset,
                 trade_rate=trade.rate,
                 fee_in_profit_currency=self.get_fee_in_profit_currency(trade),
+                fee_currency=trade.fee_currency,
+                fee_amount=trade.fee,
                 timestamp=trade.timestamp,
             )
         elif trade.trade_type == TradeType.SELL:
@@ -649,6 +655,8 @@ def _process_action(
                 receiving_amount=None,
                 gain_in_profit_currency=gain_in_profit_currency,
                 total_fee_in_profit_currency=fee_in_profit_currency,
+                fee_currency=trade.fee_currency,
+                fee_amount=trade.fee,
                 rate_in_profit_currency=selling_asset_rate,
                 timestamp=trade.timestamp,
                 loan_settlement=True,
diff --git a/rotkehlchen/accounting/events.py b/rotkehlchen/accounting/events.py
--- a/rotkehlchen/accounting/events.py
+++ b/rotkehlchen/accounting/events.py
@@ -123,6 +123,8 @@ def handle_prefork_asset_buys(
                 paid_with_asset=paid_with_asset,
                 trade_rate=trade_rate,
                 fee_in_profit_currency=fee_in_profit_currency,
+                fee_currency=None,
+                fee_amount=Fee(ZERO),
                 timestamp=timestamp,
                 is_virtual=True,
                 is_from_prefork_virtual_buy=True,
@@ -137,6 +139,8 @@ def handle_prefork_asset_buys(
                 paid_with_asset=paid_with_asset,
                 trade_rate=trade_rate,
                 fee_in_profit_currency=fee_in_profit_currency,
+                fee_currency=None,
+                fee_amount=Fee(ZERO),
                 timestamp=timestamp,
                 is_virtual=True,
                 is_from_prefork_virtual_buy=True,
@@ -148,6 +152,8 @@ def handle_prefork_asset_buys(
                 paid_with_asset=paid_with_asset,
                 trade_rate=trade_rate,
                 fee_in_profit_currency=fee_in_profit_currency,
+                fee_currency=None,
+                fee_amount=Fee(ZERO),
                 timestamp=timestamp,
                 is_virtual=True,
                 is_from_prefork_virtual_buy=True,
@@ -162,6 +168,8 @@ def handle_prefork_asset_buys(
                 paid_with_asset=paid_with_asset,
                 trade_rate=trade_rate,
                 fee_in_profit_currency=fee_in_profit_currency,
+                fee_currency=None,
+                fee_amount=Fee(ZERO),
                 timestamp=timestamp,
                 is_virtual=True,
                 is_from_prefork_virtual_buy=True,
@@ -213,6 +221,8 @@ def add_buy_and_corresponding_sell(
             paid_with_asset: Asset,
             trade_rate: FVal,
             fee_in_profit_currency: Fee,
+            fee_currency: Optional[Asset],
+            fee_amount: Optional[Fee],
             timestamp: Timestamp,
     ) -> None:
         """
@@ -232,6 +242,8 @@ def add_buy_and_corresponding_sell(
             paid_with_asset=paid_with_asset,
             trade_rate=trade_rate,
             fee_in_profit_currency=fee_in_profit_currency,
+            fee_currency=fee_currency,
+            fee_amount=fee_amount,
             timestamp=timestamp,
             is_virtual=False,
         )
@@ -290,7 +302,9 @@ def add_buy_and_corresponding_sell(
             receiving_amount=receiving_amount,
             rate_in_profit_currency=rate_in_profit_currency,
             gain_in_profit_currency=gain_in_profit_currency,
-            total_fee_in_profit_currency=fee_in_profit_currency,
+            total_fee_in_profit_currency=Fee(ZERO),  # do not pay double fees
+            fee_currency=None,
+            fee_amount=Fee(ZERO),
             timestamp=timestamp,
             is_virtual=True,
         )
@@ -303,6 +317,8 @@ def add_buy(
             paid_with_asset: Asset,
             trade_rate: FVal,
             fee_in_profit_currency: Fee,
+            fee_currency: Optional[Asset],
+            fee_amount: Optional[Fee],
             timestamp: Timestamp,
             is_virtual: bool = False,
             is_from_prefork_virtual_buy: bool = False,
@@ -356,6 +372,13 @@ def add_buy(
             rate=buy_rate,
             fee_in_profit_currency=fee_in_profit_currency,
         )
+        if fee_currency is not None and not fee_currency.is_fiat() and fee_amount is not None:
+            self.cost_basis.calculate_spend_cost_basis(
+                spending_amount=fee_amount,
+                spending_asset=fee_currency,
+                timestamp=timestamp,
+            )
+
         if timestamp >= self.query_start_ts:
             self.csv_exporter.add_buy(
                 location=location,
@@ -380,6 +403,8 @@ def add_sell_and_corresponding_buy(
             receiving_amount: FVal,
             gain_in_profit_currency: FVal,
             total_fee_in_profit_currency: Fee,
+            fee_currency: Optional[Asset],
+            fee_amount: Optional[Fee],
             trade_rate: FVal,
             rate_in_profit_currency: FVal,
             timestamp: Timestamp,
@@ -417,6 +442,8 @@ def add_sell_and_corresponding_buy(
             receiving_amount=receiving_amount,
             gain_in_profit_currency=gain_in_profit_currency,
             total_fee_in_profit_currency=total_fee_in_profit_currency,
+            fee_currency=fee_currency,
+            fee_amount=fee_amount,
             rate_in_profit_currency=rate_in_profit_currency,
             timestamp=timestamp,
             is_virtual=False,
@@ -436,7 +463,9 @@ def add_sell_and_corresponding_buy(
             bought_amount=receiving_amount,
             paid_with_asset=selling_asset,
             trade_rate=1 / trade_rate,
-            fee_in_profit_currency=total_fee_in_profit_currency,
+            fee_in_profit_currency=Fee(ZERO),  # do not count fee twice
+            fee_currency=None,
+            fee_amount=Fee(ZERO),
             timestamp=timestamp,
             is_virtual=True,
         )
@@ -450,6 +479,8 @@ def add_sell(
             receiving_amount: Optional[FVal],
             gain_in_profit_currency: FVal,
             total_fee_in_profit_currency: Fee,
+            fee_currency: Optional[Asset],
+            fee_amount: Optional[Fee],
             rate_in_profit_currency: FVal,
             timestamp: Timestamp,
             loan_settlement: bool = False,
@@ -503,6 +534,12 @@ def add_sell(
             fee_in_profit_currency=total_fee_in_profit_currency,
             gain_in_profit_currency=gain_in_profit_currency,
         )
+        if fee_currency is not None and not fee_currency.is_fiat() and fee_amount is not None:
+            self.cost_basis.calculate_spend_cost_basis(
+                spending_amount=fee_amount,
+                spending_asset=fee_currency,
+                timestamp=timestamp,
+            )
         self.handle_prefork_asset_sells(selling_asset, selling_amount, timestamp)
 
         # now search the acquisitions for `paid_with_asset` and calculate profit/loss


Test patch:
diff --git a/rotkehlchen/tests/api/test_history.py b/rotkehlchen/tests/api/test_history.py
--- a/rotkehlchen/tests/api/test_history.py
+++ b/rotkehlchen/tests/api/test_history.py
@@ -126,11 +126,12 @@ def test_query_history(rotkehlchen_api_server_with_exchanges):
     assert msg in warnings[12]
 
     errors = rotki.msg_aggregator.consume_errors()
-    assert len(errors) == 4
+    assert len(errors) == 5
     assert 'bittrex trade with unprocessable pair %$#%$#%#$%' in errors[0]
     assert 'kraken trade with unprocessable pair IDONTEXISTZEUR' in errors[1]
     assert 'kraken trade with unprocessable pair %$#%$#%$#%$#%$#%' in errors[2]
     assert 'No documented acquisition found for RDN(0x255Aa6DF07540Cb5d3d297f0D0D4D84cb52bc8e6) before' in errors[3]  # noqa: E501
+    assert 'No documented acquisition found for RDN(0x255Aa6DF07540Cb5d3d297f0D0D4D84cb52bc8e6) before' in errors[4]  # noqa: E501
 
 
 @pytest.mark.parametrize(
diff --git a/rotkehlchen/tests/integration/test_end_to_end_tax_report.py b/rotkehlchen/tests/integration/test_end_to_end_tax_report.py
--- a/rotkehlchen/tests/integration/test_end_to_end_tax_report.py
+++ b/rotkehlchen/tests/integration/test_end_to_end_tax_report.py
@@ -392,9 +392,9 @@ def test_end_to_end_tax_report(accountant):
     # action seen in history before end_ts
     assert accountant.currently_processing_timestamp == 1511626623
     general_trade_pl = FVal(result['general_trade_profit_loss'])
-    assert general_trade_pl.is_close('5032.272455644')
+    assert general_trade_pl.is_close('5033.312065244648')
     taxable_trade_pl = FVal(result['taxable_trade_profit_loss'])
-    assert taxable_trade_pl.is_close('3954.996939709')
+    assert taxable_trade_pl.is_close('3956.21087022028')
     loan_profit = FVal(result['loan_profit'])
     assert loan_profit.is_close('0.116193915')
     settlement_losses = FVal(result['settlement_losses'])
@@ -446,9 +446,9 @@ def test_end_to_end_tax_report_in_period(accountant):
     assert accountant.currently_processing_timestamp == 1511626623
     result = result['overview']
     general_trade_pl = FVal(result['general_trade_profit_loss'])
-    assert general_trade_pl.is_close('1506.937290335')
+    assert general_trade_pl.is_close('1506.698795886789')
     taxable_trade_pl = FVal(result['taxable_trade_profit_loss'])
-    assert taxable_trade_pl.is_close('642.7084519791')
+    assert taxable_trade_pl.is_close('643.1971824900041')
     loan_profit = FVal(result['loan_profit'])
     assert loan_profit.is_close('0.1140477')
     settlement_losses = FVal(result['settlement_losses'])
diff --git a/rotkehlchen/tests/unit/test_accounting.py b/rotkehlchen/tests/unit/test_accounting.py
--- a/rotkehlchen/tests/unit/test_accounting.py
+++ b/rotkehlchen/tests/unit/test_accounting.py
@@ -10,6 +10,7 @@
 from rotkehlchen.tests.utils.constants import A_DASH
 from rotkehlchen.tests.utils.history import prices
 from rotkehlchen.typing import AssetMovementCategory, EthereumTransaction, Fee, Location, Timestamp
+from rotkehlchen.utils.misc import timestamp_to_date
 
 DUMMY_ADDRESS = '0x0'
 DUMMY_HASH = b''
@@ -62,8 +63,8 @@
 @pytest.mark.parametrize('mocked_price_queries', [prices])
 def test_simple_accounting(accountant):
     accounting_history_process(accountant, 1436979735, 1495751688, history1)
-    assert accountant.general_trade_pl.is_close('557.5284549025')
-    assert accountant.taxable_trade_pl.is_close('557.5284549025')
+    assert accountant.general_trade_pl.is_close('558.25365490257463')
+    assert accountant.taxable_trade_pl.is_close('558.25365490257463')
 
 
 @pytest.mark.parametrize('mocked_price_queries', [prices])
@@ -106,11 +107,11 @@ def test_selling_crypto_bought_with_crypto(accountant):
     assert sells[0].amount == FVal('0.3853125')
     assert sells[0].timestamp == 1449809536
     assert sells[0].rate.is_close(FVal('386.0340632603'))
-    assert sells[0].fee_rate.is_close(FVal('0.96508515815085'))
+    assert sells[0].fee_rate == Fee(ZERO)  # the fee should not be double counted
     assert sells[0].gain.is_close(FVal('148.74375'))
 
-    assert accountant.general_trade_pl.is_close('73.8225270636')
-    assert accountant.taxable_trade_pl.is_close('73.8225270636')
+    assert accountant.general_trade_pl.is_close('74.1943864386100625')
+    assert accountant.taxable_trade_pl.is_close('74.1943864386100625')
 
 
 @pytest.mark.parametrize('mocked_price_queries', [prices])
@@ -362,8 +363,8 @@ def test_nocrypto2crypto(accountant):
 }])
 def test_no_taxfree_period(accountant):
     accounting_history_process(accountant, 1436979735, 1519693374, history5)
-    assert accountant.general_trade_pl.is_close('265250.9620977')
-    assert accountant.taxable_trade_pl.is_close('265250.9620977')
+    assert accountant.general_trade_pl.is_close('265251.6872977225746')
+    assert accountant.taxable_trade_pl.is_close('265251.6872977225746')
 
 
 @pytest.mark.parametrize('mocked_price_queries', [prices])
@@ -372,7 +373,7 @@ def test_no_taxfree_period(accountant):
 }])
 def test_big_taxfree_period(accountant):
     accounting_history_process(accountant, 1436979735, 1519693374, history5)
-    assert accountant.general_trade_pl.is_close('265250.9620977')
+    assert accountant.general_trade_pl.is_close('265251.6872977225746375')
     assert accountant.taxable_trade_pl.is_close('0')
 
 
@@ -494,7 +495,7 @@ def test_ignored_assets(accountant):
         'location': 'kraken',
     }]
     result = accounting_history_process(accountant, 1436979735, 1519693374, history)
-    assert FVal(result['overview']['total_taxable_profit_loss']).is_close('557.5284549025')
+    assert FVal(result['overview']['total_taxable_profit_loss']).is_close('558.253654902574637500')
 
 
 @pytest.mark.parametrize('mocked_price_queries', [prices])
@@ -537,7 +538,7 @@ def test_settlement_buy(accountant):
         1519693374,
         history,
     )
-    assert accountant.events.cost_basis.get_calculated_asset_amount('BTC').is_close('3.9920725')
+    assert accountant.events.cost_basis.get_calculated_asset_amount('BTC').is_close('3.9908725')
     assert FVal(result['overview']['total_taxable_profit_loss']).is_close('1932.598999')
     assert FVal(result['overview']['settlement_losses']).is_close('8.357159475')
 
@@ -594,7 +595,7 @@ def test_margin_events_affect_gained_lost_amount(accountant):
         history,
         margin_list=margin_history,
     )
-    assert accountant.events.cost_basis.get_calculated_asset_amount('BTC').is_close('3.748')
+    assert accountant.events.cost_basis.get_calculated_asset_amount('BTC').is_close('3.7468')
     assert FVal(result['overview']['general_trade_profit_loss']).is_close('1940.9561588')
     assert FVal(result['overview']['margin_positions_profit_loss']).is_close('-162.18738')
     assert FVal(result['overview']['total_taxable_profit_loss']).is_close('1778.7687788')
@@ -843,10 +844,72 @@ def test_sell_fiat_for_crypto(accountant):
         1519693374,
         history,
     )
-    assert FVal(result['overview']['total_profit_loss']) == FVal(25000 - 0.02 - 250 * 1.001 - 0.01)
+    assert FVal(result['overview']['total_profit_loss']) == FVal(25000 - 0.02 - 250 * 1.001)  # the 0.02/2 fee from sell of CHF to ETH is not counted since the first sell is skipped, only the buy of ETH is counted  # noqa: E501
     assert accountant.events.cost_basis.get_calculated_asset_amount(A_ETH) == FVal(1)
     assert accountant.events.cost_basis.get_calculated_asset_amount(A_BTC) == FVal(4)
     warnings = accountant.msg_aggregator.consume_warnings()
     assert len(warnings) == 0
     errors = accountant.msg_aggregator.consume_errors()
     assert len(errors) == 0
+
+
+@pytest.mark.parametrize('mocked_price_queries', [prices])
+def test_fees_count_in_cost_basis(accountant):
+    """
+    Test for https://github.com/rotki/rotki/issues/2744
+    Make sure that asset amounts used in fees are reduced.
+    """
+    history = [{
+        'timestamp': 1609537953,
+        'base_asset': 'ETH',
+        'quote_asset': 'EUR',
+        'trade_type': 'buy',
+        'rate': 598.26,
+        'fee': 1,
+        'fee_currency': 'EUR',
+        'amount': 1,
+        'location': 'kraken',
+    }, {
+        'timestamp': 1624395186,
+        'base_asset': 'ETH',
+        'quote_asset': 'EUR',
+        'trade_type': 'sell',
+        'rate': 1862.06,
+        'fee': 0.5,
+        'fee_currency': 'ETH',
+        'amount': 0.5,
+        'location': 'kraken',
+    }, {
+        'timestamp': 1625001464,
+        'base_asset': 'ETH',
+        'quote_asset': 'EUR',
+        'trade_type': 'sell',
+        'rate': 1837.31,
+        'fee': 0,
+        'fee_currency': 'EUR',
+        'amount': 0.5,
+        'location': 'kraken',
+    }]
+    result = accounting_history_process(
+        accountant,
+        1436979735,
+        1625001466,
+        history,
+    )
+
+    assert FVal(result['overview']['total_profit_loss']) == FVal(
+        # first sell also has a huge fee, same as the amount sold
+        1862.06 * 0.5 - (0.5 * 598.26 + 0.5) - 0.5 * 1862.06 +
+        # 2nd sell can't find ETH due to fee of previous so considers pure profit
+        1837.31 * 0.5,
+    )
+    assert accountant.events.cost_basis.get_calculated_asset_amount(A_ETH) is None
+    error = (
+        f'No documented acquisition found for ETH(Ethereum) before '
+        f'{timestamp_to_date(1625001464, treat_as_local=True)}. '
+        f'Let rotki know how you acquired it via a ledger action'
+    )
+    warnings = accountant.msg_aggregator.consume_warnings()
+    assert len(warnings) == 0
+    errors = accountant.msg_aggregator.consume_errors()
+    assert errors == [error]
diff --git a/rotkehlchen/tests/utils/history.py b/rotkehlchen/tests/utils/history.py
--- a/rotkehlchen/tests/utils/history.py
+++ b/rotkehlchen/tests/utils/history.py
@@ -121,6 +121,9 @@
             1539713117: FVal(178.615),
             1539713237: FVal(178.615),
             1539713238: FVal(178.615),
+            1609537953: FVal(598.26),
+            1624395186: FVal(1862.06),
+            1625001464: FVal(1837.31),
         },
     },
     strethaddress_to_identifier('0x255Aa6DF07540Cb5d3d297f0D0D4D84cb52bc8e6'): {