Repository: saulpw/visidata

Issue:
[missing option] Stack trace for disp_bool_fmt
**Small description**
Missing a  disp_bool_fmt

**Expected result**
No stack trace.

**Actual result with screenshot**
```
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/visidata/column.py", line 247, in getCell
    dw.display = self.format(typedval) or ''
  File "/usr/lib/python3.8/site-packages/visidata/column.py", line 140, in format
    return getType(self.type).formatter(self.fmtstr, typedval)
  File "/usr/lib/python3.8/site-packages/visidata/_types.py", line 40, in numericFormatter
    fmtstr = fmtstr or options['disp_'+type(typedval).__name__+'_fmt']
  File "/usr/lib/python3.8/site-packages/visidata/settings.py", line 199, in __getitem__
    vd.error('no option "%s"' % k)
  File "/usr/lib/python3.8/site-packages/visidata/statusbar.py", line 55, in error
    raise ExpectedException(args[0] if args else '')
visidata.errors.ExpectedException: no option "disp_bool_fmt"
```
**Steps to reproduce with sample data and a .vd**
```
echo '{"b":true}' | vd -f json
```

**Additional context**
Please include the version of VisiData.  latest version from develop



Discussion:
Thank you for the bug report! This was a really good use-case to find.

The tricky part about this is that there isn't really a boolean type (except in the **Options Sheet**, and they have special properties there).

 I am still not really sure how this should be handled. If we coerce it to a `str`, we will lose round-trip (it will get saved as `'true'`).

That is why a few days had passed for seemingly such a simple issue. =)
Thank you for tracking that down, and explaining why it is not such a simple issue.  I think something changed recently (last week or two)  in Visidata.  This just started happening with JSON data I have been viewing for more than a few weeks.

I was unclear on your answer that there isn't really a boolean type.  I read that as it is not one of the default Visidata types set via commands.  JSON and Python both have a boolean type, and Visidata does support JSON data, and viewing Python objects.  So it seems like there is some basic support.
And boolean data is displays correctly, but now it has these stack traces.  

However, this is one of the areas where my understanding of Visidata's representation of data is hazy.  (I am looking forward your blog post mentioned in https://github.com/saulpw/visidata/issues/580).  

Another thing I was surprised by was how many stack traces showed up.  Only loading a single row/column/cell table, there are 10 to 15 stack traces for the same thing.  That is all before doing anything else. 

@frosencrantz This is because Saul recently added options for formatting numeric data, and the default formatter became the numeric formatter. I.e. if the column does not describe a list, dictionary, or string, it will fallback to the numeric formatter as the default formatter. The numeric formatter grabs the type, and looks for the appropriate formatting option. In this case `disp_bool_fmt`.

https://github.com/saulpw/visidata/commit/70e837dbca12f4df56485a7a5da0a0483e78439b

VisiData never supported a "boolean column type". As in, before this change, I believe if you saved that jsonl, you will notice that the re-saved `True` will be covered by quotation marks. It will have essentially been coerced into a string.

That blog post is still in active development! Recent events have made it hard for me to work on most challenging things. =( This is also why I am having trouble making any sort of decision for this issue.

The formatter is called for the drawing of every single cell. That is why you see so many calls for it.
There is an additional level of difficulty! `bool` types outside of the **OptionsSheet** have never really been supported.

I have a PR with a naive simple fix that will let you pass in a file with `bool` types, and save it, but you will have some frustration editing them. (I.e. you need to pass a blank input to get `False` because inputs are being interpreted as strings.) It will not be as nice as with the **OptionsSheet**. This might be our best option for now.
After talking with Saul, he pointed out that this stacktrace could easily appear for another type that we have not considered. And that does not feel good.

So we are going to need to come up with an approach bigger than catching the `bool` case.
I hope you are feeling better. Recent events have been really unsettling.

I do have a couple questions/notes:
What is the other type that can have a stack trace?

I did save json with boolean data at the commit before https://github.com/saulpw/visidata/commit/70e837dbca12f4df56485a7a5da0a0483e78439b and it worked correctly, it did not coerce the value into a string, which may be surprising.

> > Another thing I was surprised by was how many stack traces showed up. Only loading a single row/column/cell table, there are 10 to 15 stack traces for the same thing. That is all before doing anything `else.`

> The formatter is called for the drawing of every single cell. That is why you see so many calls for it.

10 to 15 stack traces for a **single cell** table  to load&display that one cell feels excessive and might be a sign there is a performance issue.  Any thoughts on why it happens so many times?   I would guess one or two just to display that single value.  It is hard for me to tell since the stacktrace doesn't provide much context of the caller(s).   My spider-sense is tingling.  (Note: I'm not a spider, so it could just be wrong.)

Thanks for thinking about this issue and trying to come up with a clean solution. 
> I did save json with boolean data at the commit before 70e837d and it worked correctly, it did not coerce the value into a string, which may be surprising.

It is not surprising! But trying modifying one of the cells to `False`.
@frosencrantz That should be one stack trace per redraw.  It redraws up to 10 times before settling into an idle mode.  It's maybe not ideal but it's only a possible performance issue with extreme abuses of the expression column (and we have copious layers of caching to deal with that).

Resolving patch:
diff --git a/visidata/_types.py b/visidata/_types.py
--- a/visidata/_types.py
+++ b/visidata/_types.py
@@ -37,13 +37,14 @@ def anytype(r=None):
 
 
 def numericFormatter(fmtstr, typedval):
-    fmtstr = fmtstr or options['disp_'+type(typedval).__name__+'_fmt']
-    if fmtstr:
+    try:
+        fmtstr = fmtstr or options['disp_'+type(typedval).__name__+'_fmt']
         if fmtstr[0] == '%':
             return locale.format_string(fmtstr, typedval, grouping=True)
         else:
             return fmtstr.format(typedval)
-    return str(typedval)
+    except ValueError:
+        return str(typedval)
 
 
 class VisiDataType:
diff --git a/visidata/settings.py b/visidata/settings.py
--- a/visidata/settings.py
+++ b/visidata/settings.py
@@ -196,7 +196,7 @@ def __setattr__(self, k, v):   # options.foo = v
     def __getitem__(self, k):      # options[k]
         opt = self._get(k, obj=self._obj)
         if not opt:
-            vd.error('no option "%s"' % k)
+            raise ValueError('no option "%s"' % k)
         return opt.value
 
     def __setitem__(self, k, v):   # options[k] = v


Test patch: