Repository: freqtrade/freqtrade
Pull request: #4289 printing json output
Adding --json flag to print json output

closes #4289

Problem statement:
Invalid JSON returned by rest_client.py
## Describe your environment

  * Operating system: Linux (Docker)
  * Python Version: 3.8.6
  * CCXT version: 1.40.99
  * Freqtrade Version: 2021.1
  
## Describe the problem:

The JSON output from `rest_client.py` is not a valid JSON.

The reason of the problem is that the output uses single quotes instead of double quotes, and also some strings should be surrounded by quotes (At least `True, False and None`, and possibly some other strings).

### Steps to reproduce:

  1. Start the bot
  2. Make sure that there are some open trades
  3. Run the `rest_client.py status` command to show the open trades, and pass the result to JQ to verify the validity of the JSON ouput
  
### Observed Results:

  * What happened? : The JSON output is not a valid JSON
  * What did you expect to happen? The JSON output to be a valid JSON.

### Relevant code exceptions or logs

Example of the error :
```
root@3fda54e2f926:/freqtrade# python3 scripts/rest_client.py --config config_binance.json status | jq .
parse error: Invalid numeric literal at line 1, column 13
```

To fix the JSON output, we need to : 
- convert the single quotes `'` to double quotes `"`
- add double quotes around the words `True`, `False` and `None`, as JSON only allow numbers to not be quoted.

Example of the fix :
```
root@3fda54e2f926:/freqtrade# python3 scripts/rest_client.py --config config_binance.json status | tr "'" "\"" | sed -e "s|True|\"True\"|g" | sed -e "s|False|\"False\"|g" | sed -e "s|None|\"None\"|g" | jq -c .
[{"trade_id":2,"pair":"XRP/USDT","is_open":"True","exchange":"binance","amount":205.5,"amount_requested":205.5,"stake_amount":99,"strategy":"ichis","timeframe":60,"fee_open":0.001,"fee_open_cost":0.09897907500000001,"fee_open_currency":"USDT","fee_close":0.001,"fee_close_cost":"None","fee_close_currency":"None","open_date_hum":"7 minutes ago","open_date":"2021-02-01 01:44:14","open_timestamp":1612143854504,"open_rate":0.48165,"open_rate_requested":0.48165,"open_trade_value":99.07805408,"close_date_hum":"None","close_date":"None","close_timestamp":"None","close_rate":"None","close_rate_requested":"None","close_profit":"None","close_profit_pct":"None","close_profit_abs":"None","profit_ratio":-0.00935377,"profit_pct":-0.94,"profit_abs":-0.92675363,"sell_reason":"None","sell_order_status":"None","stop_loss_abs":0.385864,"stop_loss_ratio":-0.2,"stop_loss_pct":-20,"stoploss_order_id":"None","stoploss_last_update":"2021-02-01 01:44:51","stoploss_last_update_timestamp":1612143891511,"initial_stop_loss_abs":0.38532000000000005,"initial_stop_loss_ratio":-0.2,"initial_stop_loss_pct":-20,"min_rate":0.47578,"max_rate":0.48233,"open_order_id":"None","stoploss_current_dist":-0.09223600000000004,"stoploss_current_dist_pct":-19.29,"stoploss_current_dist_ratio":-0.19292198,"stoploss_entry_dist":-19.86229713,"stoploss_entry_dist_ratio":-0.20047121,"base_currency":"USDT","current_profit":-0.00935377,"current_profit_abs":-0.92675363,"current_profit_pct":-0.94,"current_rate":0.4781,"open_order":"None"}]

```

Another example with the command `stats` :
```
root@3fda54e2f926:/freqtrade# python3 scripts/rest_client.py --config config_binance.json stats | jq .
parse error: Invalid numeric literal at line 1, column 16
```

With the fix :
```
root@3fda54e2f926:/freqtrade# python3 scripts/rest_client.py --config config_binance.json stats | tr "'" "\"" | jq -c .
{"sell_reasons":{"trailing_stop_loss":{"wins":1,"losses":0,"draws":0}},"durations":{"wins":"1307.344079","draws":"N/A","losses":"N/A"}}
```

Hints from issue discussion:
the rest-client is not intended to be used by scripts, and it relies on having basically no dependencies (other than requests) to print.

It uses the `print()` command on the response dictionaries to print, which uses single quotes, and i would not know how to change this behaviour of print.

``` python
print({"asdf": "aaa"})
```

I'll mark this as "won't fix" - if you want this changed, best submit an issue against Python to change how the print command works (although i doubt they'll accept that as a valid issue).

You can create another script that does that for you, here is an example:

```
import sys, ast, json

list = ""
for line in sys.stdin:
	list += line.rstrip()

if len(list) > 0:
	l = ast.literal_eval(list)
	print(json.dumps(l))
```
You pipe rest api result to this script and it does the rest for you
this will convert python printed list automatically to json.
Simply adding quotes to None True False, can make you unable to discern a string with False value from a bool
you can also submit a pull request to address this eventually, if it's a clean, quick solution i'll for sure not argue that.

Commit message: Update rest_client.py

Patch to review:
diff --git a/scripts/rest_client.py b/scripts/rest_client.py
index 2232b842122..b214acda3f4 100755
--- a/scripts/rest_client.py
+++ b/scripts/rest_client.py
@@ -331,6 +331,13 @@ def add_arguments():
                         default=[]
                         )
 
+    parser.add_argument('--json',
+                        help='Output data in json form rather than a python dict',
+                        dest='json',
+                        action='store_true',
+                        default=False
+                        )
+
     args = parser.parse_args()
     return vars(args)
 
@@ -379,7 +386,11 @@ def main(args):
         print_commands()
         return
 
-    print(getattr(client, command)(*args["command_arguments"]))
+    output = getattr(client, command)(*args["command_arguments"])
+    if args['json']:
+        print(json.dumps(output))
+    else:
+        print(output)
 
 
 if __name__ == "__main__":


Code review comments:
diff_hunk: @@ -379,7 +386,11 @@ def main(args):
         print_commands()
         return
 
-    print(getattr(client, command)(*args["command_arguments"]))
+    output = getattr(client, command)(*args["command_arguments"])
+    if args['json']:
+        print(json.dumps(output))
+    else:
+        print(output)
path: scripts/rest_client.py
text: @user1:
hm, using json dumps is a good idea for this - but honestly, do we really need it optional?

having it in "python print" way was just lazyness, not something that's required for any functionality i'd think.

@author:
So we can just print using json by default. I'll update the pull