Repository: rotki/rotki

Issue:
Allow customization of numerical displays
## Abstract

People from different countries expect numbers to be displayed in different ways. Different thousands separator symbol, different decimal point symbol, different location for the currency symbol.

Examples:

:de:  -> 500.000,55€
:us:  -> €500,000.55

We should offer customization of number displays in the frontend.
## Task

- [ ] Add setting to allow customization of decimal point symbol
- [ ] Add setting to allow customization of thousands separator symbol
- [ ] Add setting to allow customization of currency symbol location
- [ ] Allow customization of the above settings from the frontend
- [ ] Display all numbers according to the settings


Discussion:
Fully agreed that we need something like this but I do not think it is a simple issue to implement (or rather, this specific proposal is, but doing it "right" isn't).

1) Eventually this should be part of a locale setting and then user override-able if desired, and that's non-trivial to implement. There are things to deal with like currency symbol placement and that currencies do not actually have the same currency precision. Some currencies have 0 decimals, such as the Japanese Yen, others have 3 (like most variants of the Dinar), whereas most have 2. Some currencies have 2 but there are local laws which say all numbers in payments and accounting must be integers (Hungarian Forint). Right now we are rounding all fiat currency representation to 2 which isn't strictly correct. In India for example they don't have the same digits per "grouping", e.g.: 
![image](https://user-images.githubusercontent.com/27592/88772848-9fc9a480-d181-11ea-9edb-7ec0d7373a5a.png)

Some additional info on this here: https://en.wikipedia.org/wiki/ISO_4217#Code_formation
and downloadable for us here: https://datahub.io/JohnSnowLabs/iso-4217-currency-codes#data-cli
In general there are libraries to deal with these things (numerals.js / numbro.js) but we'd need to investigate how nicely they can play with BigNumber.js. I did a little digging and also found this: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat which is relatively new and is a JS library that is built into newer versions of JS that supports internationalization of time+date+numbers.

2) Right now since we only display numbers in a certain way (most importantly the decimal symbol), it's somewhat clear how we expect numbers to be input as well. However if we display things in a user-defined format, we should also expect that users may input numbers in that same format (or they may not): e.g. what happens if a user enters a manual balance/trade and for amount they put "22.500" instead of "22,500" or "22500"? 
@isidorosp Setting this according to a locale is another step. We don't need to go for the perfect solution to the issue from the get go.

Offering the customization I mentioned above should be a good way to start.

> Right now since we only display numbers in a certain way (most importantly the decimal symbol), 

That's true, but as I said, step by step. can't offer everything at one go.

Resolving patch:
diff --git a/rotkehlchen/api/v1/encoding.py b/rotkehlchen/api/v1/encoding.py
--- a/rotkehlchen/api/v1/encoding.py
+++ b/rotkehlchen/api/v1/encoding.py
@@ -690,6 +690,16 @@ class ModifiableSettingsSchema(Schema):
     main_currency = FiatAssetField(missing=None)
     # TODO: Add some validation to this field
     date_display_format = fields.String(missing=None)
+    thousand_separator = fields.String(missing=None)
+    decimal_separator = fields.String(missing=None)
+    currency_location = fields.String(
+        strict=True,
+        validate=webargs.validate.Regexp(
+            regex=r"after|before",
+            error='Wrong currency location, should be: `after` or `before`',
+        ),
+        missing=None,
+    )
     kraken_account_type = KrakenAccountTypeField(missing=None)
     active_modules = fields.List(fields.String(), missing=None)
     frontend_settings = fields.String(missing=None)
@@ -726,6 +736,9 @@ def transform_data(  # pylint: disable=no-self-use
             eth_rpc_endpoint=data['eth_rpc_endpoint'],
             main_currency=data['main_currency'],
             date_display_format=data['date_display_format'],
+            thousand_separator=data['thousand_separator'],
+            decimal_separator=data['decimal_separator'],
+            currency_location=data['currency_location'],
             submit_usage_analytics=data['submit_usage_analytics'],
             kraken_account_type=data['kraken_account_type'],
             active_modules=data['active_modules'],
diff --git a/rotkehlchen/db/settings.py b/rotkehlchen/db/settings.py
--- a/rotkehlchen/db/settings.py
+++ b/rotkehlchen/db/settings.py
@@ -21,6 +21,9 @@
 DEFAULT_BALANCE_SAVE_FREQUENCY = 24
 DEFAULT_MAIN_CURRENCY = A_USD
 DEFAULT_DATE_DISPLAY_FORMAT = '%d/%m/%Y %H:%M:%S %Z'
+DEFAULT_THOUSAND_SEPARATOR = ','
+DEFAULT_DECIMAL_SEPARATOR = '.'
+DEFAULT_CURRENCY_LOCATION = 'after'
 DEFAULT_SUBMIT_USAGE_ANALYTICS = True
 DEFAULT_KRAKEN_ACCOUNT_TYPE = KrakenAccountType.STARTER
 DEFAULT_ACTIVE_MODULES = ['makerdao_dsr', 'makerdao_vaults', 'aave']
@@ -42,6 +45,9 @@ class DBSettings(NamedTuple):
     eth_rpc_endpoint: str = 'http://localhost:8545'
     main_currency: Asset = DEFAULT_MAIN_CURRENCY
     date_display_format: str = DEFAULT_DATE_DISPLAY_FORMAT
+    thousand_separator: str = DEFAULT_THOUSAND_SEPARATOR
+    decimal_separator: str = DEFAULT_DECIMAL_SEPARATOR
+    currency_location: str = DEFAULT_CURRENCY_LOCATION
     last_balance_save: Timestamp = Timestamp(0)
     submit_usage_analytics: bool = DEFAULT_SUBMIT_USAGE_ANALYTICS
     kraken_account_type: KrakenAccountType = DEFAULT_KRAKEN_ACCOUNT_TYPE
@@ -61,6 +67,9 @@ class ModifiableDBSettings(NamedTuple):
     eth_rpc_endpoint: Optional[str] = None
     main_currency: Optional[Asset] = None
     date_display_format: Optional[str] = None
+    thousand_separator: Optional[str] = None
+    decimal_separator: Optional[str] = None
+    currency_location: Optional[str] = None
     submit_usage_analytics: Optional[bool] = None
     kraken_account_type: Optional[KrakenAccountType] = None
     active_modules: Optional[List[str]] = None
@@ -147,6 +156,12 @@ def db_settings_from_dict(
             specified_args[key] = read_boolean(value)
         elif key == 'date_display_format':
             specified_args[key] = str(value)
+        elif key == 'thousand_separator':
+            specified_args[key] = str(value)
+        elif key == 'decimal_separator':
+            specified_args[key] = str(value)
+        elif key == 'currency_location':
+            specified_args[key] = str(value)
         elif key == 'premium_should_sync':
             specified_args[key] = read_boolean(value)
         elif key == 'last_write_ts':


Test patch:
diff --git a/frontend/app/tests/e2e/pages/account-balances-page.ts b/frontend/app/tests/e2e/pages/account-balances-page.ts
--- a/frontend/app/tests/e2e/pages/account-balances-page.ts
+++ b/frontend/app/tests/e2e/pages/account-balances-page.ts
@@ -2,6 +2,10 @@ import { capitalize } from '@/filters';
 import { ApiManualBalance } from '../../../src/services/types-api';
 import { bigNumberify, Zero } from '../../../src/utils/bignumbers';
 
+function formatAmount(amount: ApiManualBalance['amount']) {
+  return bigNumberify(amount).toFormat(2);
+}
+
 export class AccountBalancesPage {
   visit() {
     cy.get('.v-app-bar__nav-icon').click();
@@ -35,7 +39,7 @@ export class AccountBalancesPage {
 
       cy.get('@row')
         .find('.manual-balances-list__amount')
-        .should('contain', bigNumberify(balance.amount).toFormat(2));
+        .should('contain', formatAmount(balance.amount));
 
       i += 1;
     }
@@ -48,7 +52,7 @@ export class AccountBalancesPage {
 
       cy.get('@row')
         .find('.manual-balances-list__amount')
-        .should('not.contain', bigNumberify(balance.amount).toFormat(2));
+        .should('not.contain', formatAmount(balance.amount));
 
       i += 1;
     }
@@ -63,7 +67,7 @@ export class AccountBalancesPage {
 
     cy.get('@row')
       .find('.manual-balances-list__amount')
-      .should('contain', bigNumberify(balance.amount).toFormat(2));
+      .should('contain', formatAmount(balance.amount));
 
     cy.get('.manual-balances-list thead').scrollIntoView();
 
diff --git a/frontend/app/tests/e2e/pages/general-settings-page.ts b/frontend/app/tests/e2e/pages/general-settings-page.ts
--- a/frontend/app/tests/e2e/pages/general-settings-page.ts
+++ b/frontend/app/tests/e2e/pages/general-settings-page.ts
@@ -43,6 +43,24 @@ export class GeneralSettingsPage {
     cy.get(`#currency__${value.toLocaleLowerCase()}`).click();
   }
 
+  setThousandSeparator(value: string) {
+    cy.get('.general-settings__fields__thousand-separator input').clear();
+    cy.get('.general-settings__fields__thousand-separator').type(value);
+    cy.get('.general-settings__fields__thousand-separator input').blur();
+  }
+
+  setDecimalSeparator(value: string) {
+    cy.get('.general-settings__fields__decimal-separator input').clear();
+    cy.get('.general-settings__fields__decimal-separator').type(value);
+    cy.get('.general-settings__fields__decimal-separator input').blur();
+  }
+
+  setCurrencyLocation(value: 'after' | 'before') {
+    cy.get('.general-settings__fields__currency-location input').clear();
+    cy.get('.general-settings__fields__currency-location').type(value);
+    cy.get('.general-settings__fields__currency-location input').blur();
+  }
+
   setBalanceSaveFrequency(value: string) {
     cy.get('.general-settings__fields__balance-save-frequency input').clear();
     cy.get('.general-settings__fields__balance-save-frequency').type(value);
@@ -126,6 +144,9 @@ export class GeneralSettingsPage {
     anonymousUsageStatistics: boolean;
     floatingPrecision: string;
     dateDisplayFormat: string;
+    thousandSeparator: string;
+    decimalSeparator: string;
+    currencyLocation: 'after' | 'before';
     currency: string;
     balanceSaveFrequency: string;
     historicDataStart: string;
@@ -156,6 +177,21 @@ export class GeneralSettingsPage {
       'have.value',
       settings.dateDisplayFormat
     );
+    cy.get('.general-settings__fields__thousand-separator input').should(
+      'have.value',
+      settings.thousandSeparator
+    );
+    cy.get('.general-settings__fields__decimal-separator input').should(
+      'have.value',
+      settings.decimalSeparator
+    );
+    cy.get('.general-settings__fields__currency-location input').should(
+      'have.length',
+      2
+    );
+    cy.get(
+      `.general-settings__fields__currency-location input[aria-checked=true]`
+    ).should('have.value', settings.currencyLocation);
     this.verifyRPCEndpoint(settings.rpcEndpoint);
   }
 
diff --git a/frontend/app/tests/e2e/specs/settings.spec.ts b/frontend/app/tests/e2e/specs/settings.spec.ts
--- a/frontend/app/tests/e2e/specs/settings.spec.ts
+++ b/frontend/app/tests/e2e/specs/settings.spec.ts
@@ -21,6 +21,9 @@ describe('Settings', () => {
     currency: 'JPY',
     balanceSaveFrequency: '48',
     dateDisplayFormat: '%d-%m-%Y %H:%M:%S %z',
+    thousandSeparator: ',',
+    decimalSeparator: '.',
+    currencyLocation: 'after' as 'after' | 'before',
     rpcEndpoint: 'http://localhost:8545'
   };
 
diff --git a/frontend/app/tests/unit/amount_formatter.spec.ts b/frontend/app/tests/unit/amount_formatter.spec.ts
new file mode 100644
--- /dev/null
+++ b/frontend/app/tests/unit/amount_formatter.spec.ts
@@ -0,0 +1,47 @@
+import { BigNumber } from 'bignumber.js';
+import { AmountFormatter } from '@/data/amount_formatter';
+import { bigNumberify } from '@/utils/bignumbers';
+
+describe('AmountFormatter', () => {
+  const converter = new AmountFormatter();
+
+  it('should handle decimal precision correctly', () => {
+    expect(
+      converter.format(bigNumberify(127.32189477847), 2, ' ', ',')
+    ).toEqual('127,32');
+    expect(
+      converter.format(bigNumberify(127.32789477847), 2, ' ', ',')
+    ).toEqual('127,32');
+    expect(
+      converter.format(
+        bigNumberify(127.32789477847),
+        2,
+        ' ',
+        ',',
+        BigNumber.ROUND_UP
+      )
+    ).toEqual('127,33');
+  });
+
+  it('should handle decimal separator correctly', () => {
+    expect(converter.format(bigNumberify(127.32), 2, ' ', ',')).toEqual(
+      '127,32'
+    );
+  });
+
+  it('should display decimals even if integer', () => {
+    expect(converter.format(bigNumberify(127), 2, ' ', ',')).toEqual('127,00');
+  });
+
+  it('should handle one thousand separator correctly', () => {
+    expect(converter.format(bigNumberify(1127.32), 2, ' ', ',')).toEqual(
+      '1 127,32'
+    );
+  });
+
+  it('should multiple thousand separator correctly', () => {
+    expect(converter.format(bigNumberify(1221127.32), 2, ' ', ',')).toEqual(
+      '1 221 127,32'
+    );
+  });
+});
diff --git a/rotkehlchen/tests/api/test_settings.py b/rotkehlchen/tests/api/test_settings.py
--- a/rotkehlchen/tests/api/test_settings.py
+++ b/rotkehlchen/tests/api/test_settings.py
@@ -73,6 +73,12 @@ def test_set_settings(rotkehlchen_api_server):
             value = '10/10/2016'
         elif setting == 'date_display_format':
             value = '%d/%m/%Y-%H:%M:%S'
+        elif setting == 'thousand_separator':
+            value = ','
+        elif setting == 'decimal_separator':
+            value = '.'
+        elif setting == 'currency_location':
+            value = 'after'
         elif setting == 'eth_rpc_endpoint':
             value = 'http://working.nodes.com:8545'
         elif setting == 'main_currency':
@@ -448,6 +454,39 @@ def test_set_settings_errors(rotkehlchen_api_server):
         status_code=HTTPStatus.BAD_REQUEST,
     )
 
+    # invalid type thousand_separator
+    data = {
+        'settings': {'thousand_separator': 124.1},
+    }
+    response = requests.put(api_url_for(rotkehlchen_api_server, "settingsresource"), json=data)
+    assert_error_response(
+        response=response,
+        contained_in_msg='Not a valid string',
+        status_code=HTTPStatus.BAD_REQUEST,
+    )
+
+    # invalid type decimal_separator
+    data = {
+        'settings': {'decimal_separator': 124.1},
+    }
+    response = requests.put(api_url_for(rotkehlchen_api_server, "settingsresource"), json=data)
+    assert_error_response(
+        response=response,
+        contained_in_msg='Not a valid string',
+        status_code=HTTPStatus.BAD_REQUEST,
+    )
+
+    # invalid type currency_location
+    data = {
+        'settings': {'currency_location': 124.1},
+    }
+    response = requests.put(api_url_for(rotkehlchen_api_server, "settingsresource"), json=data)
+    assert_error_response(
+        response=response,
+        contained_in_msg='Not a valid string',
+        status_code=HTTPStatus.BAD_REQUEST,
+    )
+
     # invalid type kraken_account_type
     data = {
         'settings': {'kraken_account_type': 124.1},
diff --git a/rotkehlchen/tests/db/test_db.py b/rotkehlchen/tests/db/test_db.py
--- a/rotkehlchen/tests/db/test_db.py
+++ b/rotkehlchen/tests/db/test_db.py
@@ -18,6 +18,9 @@
     DEFAULT_ANONYMIZED_LOGS,
     DEFAULT_BALANCE_SAVE_FREQUENCY,
     DEFAULT_DATE_DISPLAY_FORMAT,
+    DEFAULT_THOUSAND_SEPARATOR,
+    DEFAULT_DECIMAL_SEPARATOR,
+    DEFAULT_CURRENCY_LOCATION,
     DEFAULT_INCLUDE_CRYPTO2CRYPTO,
     DEFAULT_INCLUDE_GAS_COSTS,
     DEFAULT_KRAKEN_ACCOUNT_TYPE,
@@ -259,6 +262,9 @@ def test_writing_fetching_data(data_dir, username):
         'main_currency': DEFAULT_MAIN_CURRENCY.identifier,
         'anonymized_logs': DEFAULT_ANONYMIZED_LOGS,
         'date_display_format': DEFAULT_DATE_DISPLAY_FORMAT,
+        'thousand_separator': DEFAULT_THOUSAND_SEPARATOR,
+        'decimal_separator': DEFAULT_DECIMAL_SEPARATOR,
+        'currency_location': DEFAULT_CURRENCY_LOCATION,
         'last_data_upload_ts': 0,
         'premium_should_sync': False,
         'submit_usage_analytics': True,
@@ -290,6 +296,9 @@ def test_settings_entry_types(database):
         eth_rpc_endpoint='http://localhost:8545',
         balance_save_frequency=24,
         date_display_format='%d/%m/%Y %H:%M:%S %z',
+        thousand_separator=',',
+        decimal_separator='.',
+        currency_location='after',
         submit_usage_analytics=False,
     ))
 
@@ -320,6 +329,12 @@ def test_settings_entry_types(database):
     assert res.anonymized_logs is True
     assert isinstance(res.date_display_format, str)
     assert res.date_display_format == '%d/%m/%Y %H:%M:%S %z'
+    assert isinstance(res.thousand_separator, str)
+    assert res.thousand_separator == ','
+    assert isinstance(res.decimal_separator, str)
+    assert res.decimal_separator == '.'
+    assert isinstance(res.currency_location, str)
+    assert res.currency_location == 'after'
     assert isinstance(res.submit_usage_analytics, bool)
     assert res.submit_usage_analytics is False
     assert isinstance(res.active_modules, list)